Add rollup_getVirtualZkEVMStateMerkleProofV0

[garyschulte]

PR Description

Adds an rpc endpoint to encapsulate the eth_simulateV1 call for a single transaction, and returns a proof of a 'virtual' block on top of the specified parent, akin to rollup_getZkEVMStateMerkleProofV0.

Fixed Issue(s)

relates to Consensys/besu-shomei-plugin#70

Documentation

• I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

• I thought about adding a changelog entry, and added one if I deemed necessary.

---

Note

Medium Risk
Adds a new RPC path that drives worldstate simulation and trace generation on cached snapshots, plus updates trielog decoding for a new Besu format; correctness and cache/overlay edge cases could affect proof outputs or runtime behavior.

Overview
Adds a new JSON-RPC method rollup_getVirtualZkEVMStateMerkleProofV0 that simulates a single transaction on top of a specified parent block (via Besu eth_simulateV1), then returns the parent/end zk state root hashes plus the generated zk state merkle proof.

To support non-persistent simulation, introduces LayeredWorldStateStorage (in-memory overlay with delete tracking) and ZkWorldStateArchive.generateVirtualTrace(...) which applies a decoded trielog against a cached parent snapshot using an ephemeral trace store. Also updates trielog decoding for forward compatibility with Besu metadata, improves JSON-RPC client error handling/modeling, bumps besuVersion, and adjusts sync queue waiting logic to poll with timeouts/backoff for cleaner shutdown behavior.

^{Written by Cursor Bugbot for commit 8ff6eac. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

[garyschulte]

when testing, I found these resources to be useful:
test genesis, toml, and 33 test blocks

and this curl to simulate a block 34 on top of the chain of 33:

curl --location 'http://localhost:8888' \
--header 'Content-Type: application/json' \
--data '{
    "jsonrpc": "2.0",
    "method": "rollup_getVirtualZkEVMStateMerkleProofV0",
    "params": [
        {
            "blockNumber": 34,
            "transaction": "0xf868808405f5e10082520894353535353535353535353535353535353535353585e8d4a510008025a0263f64b115ad522b540585a37302fc710dd5d48702f93420abd0e03c4fa12e4da0711c61d2cfd3fc6efd6cd9a8a6dbd6a873e12b74801980614ac7e7a3d2117d7e"
        }
    ],
    "id": 1
}
'

[matkt]

do we need also to use getValidParentNode in this case ?

because it seems we are not checking if the key returned by the overlay is deleted

final TrieStorage.Range overlayRange = overlayStorage.getNearestKeys(hkey);

[garyschulte]

We shouldn't need getValidParentNode in that case because we only exact match for center nodes. Line 120 we filter parent range center node by deleted keys.

[matkt]

why not managing the delete in the InMemoryWorldStateStorage ?

[garyschulte]

I was reluctant to change InMemoryWorldStateStorage - if you want to change this to use the Besu approach that sounds reasonable. I think the complexity is the same either way since we still have to defer to the parent for getNearest - that is where the complexity is. The unit tests assert correct behavior (on toy state), so we should have good coverage for a refactor

[matkt]

why not doing something similar to the LayeredKeyValueStorage besu implementation. this way to have InMemoryWorldStateStorage as a field seems to add a lot of complexity in the code

in Besu LayeredKeyValueStorage extend directly a InMemoryWorldStateStorage version that track deletion and we only have the parent