feat(engine): share sparse trie pipeline with payload builder by joshieDo · Pull Request #23246 · paradigmxyz/reth

joshieDo · 2026-03-26T18:06:15Z

#23242 follow-up

Shares the engine's incremental sparse trie pipeline with the payload builder so FCU-triggered block builds skip the blocking state_root_with_updates() call.

Moved & unified types

StateRootHandle (new, in reth-trie-parallel::state_root_task):

Replaces engine-local StateRootHandle (was in payload_processor/mod.rs)
Single type used by both engine (newPayload) and payload builder (FCU)

Also moved to state_root_task:

MultiProofMessage, Source, StateRootComputeOutcome, StateHookSender, evm_state_to_hashed_post_state()

`BlockBuilder::finish()`

finish(provider, None) — computes state root internally (existing behavior)
finish(provider, Some((root, trie_updates))) — uses pre-computed root

Payload builder wiring

In default_ethereum_payload:

Sets state hook via handle.state_hook() before tx loop
After execution, drops hook → FinishedStateUpdates, recvs root, calls finish with pre-computed result
Falls back to sync path when handle is None

Engine convergence

spawn_sparse_trie_handle deleted — spawn_state_root now takes halve_workers: bool
EngineValidator::sparse_trie_handle_for returns Option<StateRootHandle>
Gated behind --engine.share-sparse-trie-with-payload-builder (default false)

Extract MultiProofMessage, Source, StateRootComputeOutcome, SparseTrieHandle, and evm_state_to_hashed_post_state into reth-trie-parallel::sparse so the payload builder can access them. Add trie_handle: Option<SparseTrieHandle> to BuildNewPayload, gated behind --engine.share-sparse-trie-with-payload-builder (default false). When enabled, the engine spawns a sparse trie pipeline at FCU time and passes the handle to the payload builder. The builder can stream state updates into the pipeline for incremental state root computation instead of blocking on state_root_with_updates(). Amp-Thread-ID: https://ampcode.com/threads/T-019d2b14-fe40-70dc-b210-33930dcc6e98 Co-authored-by: Amp <amp@ampcode.com>

github-actions · 2026-03-26T18:07:00Z

⚠️ Changelog not found.

A changelog entry is required before merging. We've generated a suggested changelog based on your changes:

Preview

---
reth-engine-primitives: minor
reth-engine-tree: minor
reth-engine-util: patch
reth-ethereum-payload-builder: minor
reth-trie-parallel: minor
reth-node-ethereum: patch
---

Added support for sharing a sparse trie pipeline with the payload builder via a new `share_sparse_trie_with_payload_builder` config option in `TreeConfig`. Moved `StateRootHandle`, `StateRootComputeOutcome`, `StateRootMessage`, `StateHookSender`, and related types from `reth-engine-tree` internals into `reth-trie-parallel` and renamed `MultiProofMessage` to `StateRootMessage` throughout.

Add changelog to commit this to your branch.

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b14-fe40-70dc-b210-33930dcc6e98 Co-authored-by: Amp <amp@ampcode.com>

Wire the engine's incremental sparse trie pipeline into the payload builder so FCU-triggered block builds can skip the blocking state_root_with_updates() call. Key changes: - Unify SparseTrieHandle into StateRootHandle (reth-trie-parallel::state_root_task) with state_hook(), state_root(), updates_tx() accessors - Merge finish/finish_with_state_root into single BlockBuilder::finish() accepting Option<(B256, TrieUpdates)> - Wire state hook in default_ethereum_payload() via handle.state_hook() - Converge spawn_sparse_trie_handle into spawn_state_root with halve_workers param - Add --engine.share-sparse-trie-with-payload-builder CLI flag (default false) - Thread StateRootHandle through BuildNewPayload → BasicPayloadJob → BuildArguments Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

… trie failure Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

yongkangc · 2026-03-27T05:26:34Z

+        let halve_workers = env.transaction_count <= Self::SMALL_BLOCK_PROOF_WORKER_TX_THRESHOLD;
+        let state_root_handle = self.spawn_state_root(
+            multiproof_provider_factory,
+            env.parent_state_root,
+            halve_workers,
+            config,
+        );


i see, we wanna half worker if its a small block.

klkvr · 2026-03-27T11:08:52Z

    fn finish(
        self,
        state: impl StateProvider,
+        state_root_precomputed: Option<(B256, TrieUpdates)>,


makes me wonder if this should just take a closure like |&db| -> (state_root, trie_updates)

this should likely allow us to upstream this abstraction into alloy-evm entirely

mattsse

this has a few drawbacks/side-effects users should be aware of which arent properly documented

the fact that this advances the trie makes this pretty much unsuitable for the ethereum implementation and we should probably not use it at all to avoid side-effects for the --always-prepare-payload

but also, this setting is opt in so likely not an issue unless misused

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`7f4a9a0...f0d07c3`](paradigmxyz/reth@7f4a9a0...f0d07c3) 🔗 Amp thread: https://ampcode.com/threads/T-019d4215-b76f-7663-b8f3-8fde1da1165c **Engine** - Share execution cache and sparse trie pipeline with payload builder ([#23242](paradigmxyz/reth#23242), [#23246](paradigmxyz/reth#23246)) - Add method to get payload resolve future ([#23256](paradigmxyz/reth#23256)) - Backpressure, take 2 ([#23280](paradigmxyz/reth#23280)) - Return -38003 for FCUv2 payloadAttributes mismatch ([#22924](paradigmxyz/reth#22924)) - Fix double decrement in account cache size ([#23249](paradigmxyz/reth#23249)) **Trie** - Call root before prune ([#23243](paradigmxyz/reth#23243)) - Use Entry API in `MultiProofTargets::extend_inner` ([#23247](paradigmxyz/reth#23247)) - Record trie cursor metrics ([#23252](paradigmxyz/reth#23252)) - Add `SparseStateTrie::update_account_stateless` for stateless validation ([#23272](paradigmxyz/reth#23272)) **Networking** - Prefer peer-reported block number in session activation ([#23275](paradigmxyz/reth#23275)) - Resolve DNS for ExternalAddr in `external_addr_with` ([#23269](paradigmxyz/reth#23269)) **Consensus** - Retry block subscription on initial connection failure ([#23233](paradigmxyz/reth#23233)) **Refactor** - Remove OP `ExecutionPayload` impl and op feature from payload-primitives ([#23253](paradigmxyz/reth#23253)) - Remove OP `PayloadAttributesBuilder` impl and op feature from engine-local ([#23255](paradigmxyz/reth#23255)) - Relax RPC converter impls ([#23254](paradigmxyz/reth#23254)) **Perf / Bench** - Use `FastInstant` for remaining metrics timing ([#23265](paradigmxyz/reth#23265)) - Add hourly main regression bench ([#23219](paradigmxyz/reth#23219)) **DB** - Add `create_test_provider_factory_with_chain_spec_and_db_args` ([#23270](paradigmxyz/reth#23270)) **CLI** - Add more WARN logging before download retries ([#23258](paradigmxyz/reth#23258)) - Use `HeaderTy` for stage dump headers ([#23274](paradigmxyz/reth#23274)) **Testing** - Add regression test for parked basefee ancestor handling ([#23277](paradigmxyz/reth#23277)) **Deps** - Bump alloy 1.8.2, alloy-evm, lighthouse v8.1.3 ([#23241](paradigmxyz/reth#23241), [#23289](paradigmxyz/reth#23289), [#23239](paradigmxyz/reth#23239)) - Weekly `cargo update` ([#23267](paradigmxyz/reth#23267)) ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-019d4215-e6e7-7542-acaa-239d595b2916 - **Reth dependency bump**: All `reth-*` crates updated from rev `7f4a9a0` to `f0d07c3`; `alloy-evm` bumped from `0.29.2` to `0.30.0` - **`reth-etl` reordered**: Moved from its previous position to sit next to `reth-trie` (no functional change) - **`tempo-contracts` feature dropped**: Removed default `features = ["serde"]` from workspace dependency - **`TransactionEnv` → `TransactionEnvMut` rename**: `reth_evm::TransactionEnv` trait was renamed to `TransactionEnvMut`; all imports and impls updated - **`TransactionEnv::nonce()` removed**: The `nonce(&self)` method was removed from the trait; call sites now use `revm::context::Transaction::nonce()` instead - **`TxResult::into_result()` added**: New required method `into_result(self) -> ResultAndState` added to the `TxResult` trait impl for `TempoTxResult` - **`BuildArguments::new` signature expanded**: Two new parameters (`None, None`) added for `execution_cache` and `trie_handle`; corresponding destructure updated to ignore them - **`builder.finish()` signature change**: Now takes an additional `Option` parameter (passed as `None`) for trie handle [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/23779717485)

Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`7f4a9a0...f8efc76`](paradigmxyz/reth@7f4a9a0...f8efc76) 🔗 Amp thread: https://ampcode.com/threads/T-019d473e-4c63-7539-bda6-d72354aae810 **Engine** - Share execution cache and sparse trie pipeline with payload builder ([#23242](paradigmxyz/reth#23242), [#23246](paradigmxyz/reth#23246)) - Add backpressure, take 2 ([#23280](paradigmxyz/reth#23280)) - Add method to get payload resolve future ([#23256](paradigmxyz/reth#23256)) - Return -38003 for FCUv2 payloadAttributes mismatch ([#22924](paradigmxyz/reth#22924)) - Fix double decrement in account cache size ([#23249](paradigmxyz/reth#23249)) **Trie** - Call root before prune ([#23243](paradigmxyz/reth#23243)) - Use Entry API in `MultiProofTargets::extend_inner` ([#23247](paradigmxyz/reth#23247)) - Record trie cursor metrics ([#23252](paradigmxyz/reth#23252)) - Add `SparseStateTrie::update_account_stateless` for stateless validation ([#23272](paradigmxyz/reth#23272)) **RPC** - Integrate `reth-rpc-traits` and remove `IntoRpcTx` ([#23288](paradigmxyz/reth#23288)) - Relax rpc converter impls ([#23254](paradigmxyz/reth#23254)) **Net** - Prefer peer-reported block number in session activation ([#23275](paradigmxyz/reth#23275)) - Retry block subscription on initial connection failure ([#23233](paradigmxyz/reth#23233)) - Resolve DNS for ExternalAddr in `external_addr_with` ([#23269](paradigmxyz/reth#23269)) **Payload / Refactor** - Remove OP `ExecutionPayload` impl and op feature from payload-primitives ([#23253](paradigmxyz/reth#23253)) - Remove OP `PayloadAttributesBuilder` impl and op feature from engine-local ([#23255](paradigmxyz/reth#23255)) - Remove changeset count APIs from storage ([#23310](paradigmxyz/reth#23310)) **DB / Storage** - Add `create_test_provider_factory_with_chain_spec_and_db_args` ([#23270](paradigmxyz/reth#23270)) - Add `reth-bb` binary with multi-segment big block execution support ([#23140](paradigmxyz/reth#23140)) - Add era type override functionality to `EraClient` ([#23307](paradigmxyz/reth#23307)) - Make snapshot API URL overridable ([#23303](paradigmxyz/reth#23303)) **Perf** - Use `FastInstant` for remaining metrics timing ([#23265](paradigmxyz/reth#23265)) **Bench / Testing** - Add hourly main regression bench ([#23219](paradigmxyz/reth#23219)) - Add regression test for parked basefee ancestor handling ([#23277](paradigmxyz/reth#23277)) **CLI** - Add more WARN logging before download retries ([#23258](paradigmxyz/reth#23258)) - Use `HeaderTy` for stage dump headers ([#23274](paradigmxyz/reth#23274)) **Deps** - Bump alloy 1.8.2, alloy-evm, Lighthouse v8.1.3 ([#23241](paradigmxyz/reth#23241), [#23289](paradigmxyz/reth#23289), [#23239](paradigmxyz/reth#23239)) - Weekly `cargo update` ([#23267](paradigmxyz/reth#23267)) **Grafana** - Add sparse trie idle metrics to overview ([#23302](paradigmxyz/reth#23302)) ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-019d473e-8aba-7283-b729-057391d70bc6 - **Reth dependency bump**: All `reth-*` crates updated from rev `7f4a9a0` to `f8efc76` - **alloy-evm version bump**: `0.29.2` → `0.30.0`; `revm-inspectors` `0.36.1` → `0.36.0` - **`TransactionEnv` → `TransactionEnvMut`**: Trait moved from `reth_evm::TransactionEnv` to `alloy_evm::TransactionEnvMut`; `nonce()` getter removed from the trait (now accessed via `revm::context::Transaction::nonce()` instead) - **`FromConsensusHeader` re-export flattened**: Import path changed from `reth_rpc_convert::transaction::FromConsensusHeader` to `reth_rpc_convert::FromConsensusHeader` - **`TxResult::into_result` added**: New required method on the `TxResult` trait to consume and return `ResultAndState` - **`BuildArguments::new` signature expanded**: Now takes two additional `None` parameters (likely cached/prebuilt payload fields) - **`BuildArguments` destructure**: Uses `..` rest pattern to ignore new fields - **`BlockBuilder::finish` signature change**: Now takes an additional parameter (`None` — likely an optional requests list) - **`snapshot_api_url` field added**: New required field on download URL config struct [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/23831300439) --------- Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com> Co-authored-by: Arsenii Kulikov <62447812+klkvr@users.noreply.github.com>

github-project-automation Bot added this to Reth Tracker Mar 26, 2026

github-project-automation Bot moved this to Backlog in Reth Tracker Mar 26, 2026

joshieDo and others added 6 commits March 26, 2026 18:09

chore: fix toml formatting

3f2752c

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b14-fe40-70dc-b210-33930dcc6e98 Co-authored-by: Amp <amp@ampcode.com>

chore: improve doc comments

d998ac8

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b14-fe40-70dc-b210-33930dcc6e98 Co-authored-by: Amp <amp@ampcode.com>

test(e2e): add test for shared sparse trie with payload builder

216aa75

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

fix(payload): install state hook before pre-exec changes, fallback on…

4ada699

… trie failure Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

chore: regenerate CLI docs, fix broken doc link

46cc89e

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

joshieDo marked this pull request as ready for review March 26, 2026 20:13

joshieDo requested review from Rjected, gakonst, klkvr, mattsse, mediocregopher, shekhirin and yongkangc as code owners March 26, 2026 20:13

yongkangc reviewed Mar 27, 2026

View reviewed changes

klkvr approved these changes Mar 27, 2026

View reviewed changes

github-project-automation Bot moved this from Backlog to In Progress in Reth Tracker Mar 27, 2026

shekhirin approved these changes Mar 27, 2026

View reviewed changes

mattsse requested changes Mar 27, 2026

View reviewed changes

joshieDo and others added 3 commits March 27, 2026 15:53

refactor: rename MultiProofMessage to StateRootMessage

7b78097

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

Merge branch 'main' into joshie/share-sparse-trie-with-payload-builder

311324c

docs: document sparse trie sharing side effects

3d1e33d

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

joshieDo requested a review from mattsse March 27, 2026 16:16

docs: add 'and cleared' to trie invalidation note

215b96f

Amp-Thread-ID: https://ampcode.com/threads/T-019d2b69-f70c-77a9-b37d-be082dd5d7a7 Co-authored-by: Amp <amp@ampcode.com>

mattsse approved these changes Mar 27, 2026

View reviewed changes

joshieDo added this pull request to the merge queue Mar 27, 2026

Merged via the queue into main with commit 29bab06 Mar 27, 2026
35 checks passed

joshieDo deleted the joshie/share-sparse-trie-with-payload-builder branch March 27, 2026 16:48

github-project-automation Bot moved this from In Progress to Done in Reth Tracker Mar 27, 2026

decofe mentioned this pull request Mar 28, 2026

deps: update reth from main (2026-03-31) tempoxyz/tempo#3332

Merged

decofe mentioned this pull request Apr 1, 2026

deps: update reth from main (2026-04-01) tempoxyz/tempo#3408

Merged

This was referenced May 27, 2026

chore: merge develop-v2.2-new into develop bnb-chain/reth#192

Open

chore: merge develop-v2.2-new into develop (conflict resolved, pin reth to de11c921) bnb-chain/reth-bsc#361

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(engine): share sparse trie pipeline with payload builder#23246

feat(engine): share sparse trie pipeline with payload builder#23246
joshieDo merged 11 commits into
mainfrom
joshie/share-sparse-trie-with-payload-builder

joshieDo commented Mar 26, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

yongkangc Mar 27, 2026

Uh oh!

klkvr Mar 27, 2026

Uh oh!

Uh oh!

mattsse left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

joshieDo commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Moved & unified types

BlockBuilder::finish()

Payload builder wiring

Engine convergence

Uh oh!

github-actions Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Changelog not found.

Uh oh!

yongkangc Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

klkvr Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattsse left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

joshieDo commented Mar 26, 2026 •

edited

Loading

`BlockBuilder::finish()`

github-actions Bot commented Mar 26, 2026 •

edited

Loading