feat(rpc): debug_executionWitness

[clabby]

Overview

Adds a new endpoint to the debug namespace for generating historical execution witnesses.

Rationale

For ZK and Optimistic proving systems, generating execution witnesses for stateless execution can be tough. Before peter's recent cross-client validation proposal, there were fairly limited proposals for this feature pre-verkle. All existing options involve expensive-to-run nodes with large storage requirements, i.e. gcmode=archive + state.scheme=hash geth.

Hash scheme geth does the trick, but in an abusive way towards their debug_dbGet endpoint, which just-so-happens to key all RLP-encoded trie nodes by their hash (H(rlp(trie_node)) -> rlp(trie_node)).

How does it work?

1. The block is re-executed on top if its parent's state.
2. All referenced accounts & their storage slots (both referenced and modified) are pulled from the wrapping CacheDB.
3. For each of these accounts, an AccountProof is generated, utilizing the HistoricalStateProvider::proof method.
4. All RLP encoded trie nodes are hashed, and placed into a map of H(rlp(trie_node)) -> rlp(trie_node).

DoS concerns

Due to reth's database format only storing merkle diffs, this process can become very expensive for the same reason that state root generation in historical execution within reth can be very expensive. Each diff is layered, which means the further back we reach, the more expensive the operation becomes (approaching upper bound, potentially unable to even fit in memory due to the sheer volume of layers.)

Because of this, this endpoint has been placed in the debug_* namespace, which is not often exposed to the open internet by node operators. Exposing this endpoint can pose major DoS concerns for node operators as it stands.

Future historical eth_getProof support

Historical eth_getProof support was already possible in reth before this PR, but was never implemented due to the above DoS concerns (per @rkrasiuk). This PR does not remove the requirement for being at tip to use eth_getProof, but this could be supported in a future PR, by adding a sane default for allowed eth_getProof depth, configurable via the CLI. A max chain depth for eth_getProof would effectively cap the number of in-memory layers that must be applied to the trie.

TODO

In order to complete this feature, we'll need to be able to also include branch sibling nodes that must be referenced in order to delete leaves in the MPT. This endpoint currently gives the entirety of the witness data required for execution, but not to re-produce the block header for the processed block. The crux of this is that we cannot know which nodes we need to reveal until we're performing the deletion, meaning we also have to compute the state root in this endpoint and collect them, as these nodes are not included within the account proofs we already generate.

To outline the problem, let's say we start with this small trie, where the root node is a branch containing 2 children (both leaf nodes):

If we want to delete the node in the 3 bucket, the branch node would be left with only one child - this is an invalid state. To complete the deletion, we must reveal A (which is hashed), and collapse the branch node into A.

In order of operation:

1. Replace 3 with rlp(empty) = 0x80
2. Reveal the preimage of H(node_at(A)), and prepend the branch nibble (A) to its path.
3. Replace the branch node with the truncated leaf.

[clabby]

reminder to self: Need 4788 call here

[Rjected]

wondering if we can eventually use the block executor traits here, as there will be more system contract calls and maintaining a copy of block execution logic in debug APIs seems brittle

[clabby]

Agree. I also specifically need this for Optimism execution, so that would be ideal. Glad to circle back around to this before we merge, just have to get it working first.

[gakonst]

This should prob be part of the Evm Executor as a execute_witnessed() style operation? and then the RPC method calls that?

[rkrasiuk]

should be access list inspector instead

[mattsse]

suggestions and nits

[Rjected]

still need to call 4788 etc here right?

[rkrasiuk]

good catch

[mattsse]

lgtm

[mattsse]

one more thing, this doesn't check if the target block is post merge

do we need to call any other systemcalls or just 4788?