refactor(provider): use overlay builders in historical state paths#23667
Merged
klkvr merged 21 commits intoApr 23, 2026
Conversation
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db016-ea05-74e4-a085-27f4173a1f9f Co-authored-by: Amp <amp@ampcode.com>
mediocregopher
added
A-engine
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db037-4b3b-77c1-9832-42d24c58560e Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db037-4b3b-77c1-9832-42d24c58560e Co-authored-by: Amp <amp@ampcode.com>
Restore the historical provider behavior to match origin/main while keeping the ChangesetCache constructor plumbing and shared reorg-validator cache wiring. Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db05b-f81c-754e-9229-81b57d6d32f5 Co-authored-by: Amp <amp@ampcode.com>
mediocregopher
commented
Apr 21, 2026
| ctx.components().evm_config().clone(), | ||
| || async { | ||
| // Create a separate cache for reorg validator (not shared with main engine) | ||
| let reorg_cache = ChangesetCache::new(); |
Member
Author
There was a problem hiding this comment.
Tangential change, but it's not necessary to have a separate ChangesetCache for debug reorgs, the cache is keyed by block hash anyway.
mediocregopher
requested review from
Rjected,
joshieDo,
klkvr,
mattsse and
shekhirin
as code owners
mediocregopher
changed the title
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db074-fac6-76d2-8950-3e84b3c895a4 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db074-fac6-76d2-8950-3e84b3c895a4 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db0ad-77b5-76ba-8b0b-6683a56f63c5 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db0ad-77b5-76ba-8b0b-6683a56f63c5 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db0ad-77b5-76ba-8b0b-6683a56f63c5 Co-authored-by: Amp <amp@ampcode.com>
Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
mediocregopher
changed the title
Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
klkvr
approved these changes
Apr 21, 2026
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db4f1-b5cb-75ee-b759-0c6badfe0ff6 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db4f1-b5cb-75ee-b759-0c6badfe0ff6 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db4f1-b5cb-75ee-b759-0c6badfe0ff6 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db4f1-b5cb-75ee-b759-0c6badfe0ff6 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db4f1-b5cb-75ee-b759-0c6badfe0ff6 Co-authored-by: Amp <amp@ampcode.com>
…ical-state-provider-changeset-cache
Member
Author
|
derek bench |
Member
Author
|
derek bench big-blocks |
Member
|
✅ Benchmark complete! View job Benchmark Results
500 blocks, 200 warmup Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
Member
|
✅ Benchmark complete! View job Benchmark Results
500 big blocks, 20 warmup Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db5d3-fb9b-7309-b8e9-dfc2ca6290a7 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db9e5-51dc-763e-b1ec-4e09fbc9e8e8 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019db9e5-51dc-763e-b1ec-4e09fbc9e8e8 Co-authored-by: Amp <amp@ampcode.com>
klkvr
approved these changes
Apr 23, 2026
klkvr
deleted the
mediocregopher/historical-state-provider-changeset-cache
branch
umeeSthein
pushed a commit
to umeeSthein/tempo
that referenced
this pull request
Apr 30, 2026
Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`7839f3d...73ec2c9`](paradigmxyz/reth@7839f3d...73ec2c9) 🔗 Amp thread: https://ampcode.com/threads/T-019dd89b-f659-75eb-a3f2-3e769d63297f **Engine** - Spawn BAL hashed state before storage prefetch ([#23761](paradigmxyz/reth#23761)) and disable BAL parallel execution by default ([#23764](paradigmxyz/reth#23764)) - Add CLI flag to disable BAL storage prefetch ([#23770](paradigmxyz/reth#23770)) - Log transient invalid header cache skips ([#23711](paradigmxyz/reth#23711)) - Add `DecodedBal` in `ExecutionEnv` ([#23675](paradigmxyz/reth#23675)) **RPC** - Clean up eth state cache reorg entries ([#23683](paradigmxyz/reth#23683)) - Pass block timestamp to txn ([#23700](paradigmxyz/reth#23700)) - Include block numbers in `BlockRangeExceedsHead` error ([#23720](paradigmxyz/reth#23720)) **Network / P2P** - Add Basic in-memory BAL store and BAL request to block access list requests ([#23682](paradigmxyz/reth#23682), [#23710](paradigmxyz/reth#23710)) - Enforce BAL response soft limit and apply count cap to handler ([#23725](paradigmxyz/reth#23725), [#23754](paradigmxyz/reth#23754)) - Avoid RLP-decoding `NewBlock` payloads ([#23712](paradigmxyz/reth#23712)) and bound memory footprint of p2p messages ([#23718](paradigmxyz/reth#23718)) - Memory-bound channel between network and tx manager ([#23802](paradigmxyz/reth#23802)) - Track unknown tx types in announcement metrics ([#23688](paradigmxyz/reth#23688)) - Retain active session buffer capacity ([#23702](paradigmxyz/reth#23702)) and respect peer requirements for fetch followups ([#23706](paradigmxyz/reth#23706)) - Support binding discv5 and discv4 to the same port ([#23613](paradigmxyz/reth#23613)) and use Weak reference in discv5 kbuckets bg task to release port on shutdown ([#23282](paradigmxyz/reth#23282)) **Trie** - Skip DB seek on exact overlay hits ([#23559](paradigmxyz/reth#23559)) - Account for heap-allocated blinded hashes in `SparseNode::memory_size` ([#23726](paradigmxyz/reth#23726)) **DB / Storage** - `reth db migrate-v2` for pruned nodes ([#23716](paradigmxyz/reth#23716)) - Pass `ExecutedBlocks` to `OverlayBuilder`, reduce reverts queried ([#23657](paradigmxyz/reth#23657)) - Add empty request check to storage values ([#23714](paradigmxyz/reth#23714)) - Derive `Eq` for `IntegerList` ([#23709](paradigmxyz/reth#23709)) and reorder unix deps after strum ([#23697](paradigmxyz/reth#23697)) **Provider / EVM** - Use overlay builders in historical state paths ([#23667](paradigmxyz/reth#23667)) - Return gas output from block builder ([#23744](paradigmxyz/reth#23744)) - Expose executor transaction result type ([#23759](paradigmxyz/reth#23759)) **Perf** - Configurable rocksdb block cache size and re-use of mdbx provider in re-execute ([#23701](paradigmxyz/reth#23701)) - Enable revm `p256-aws-lc-rs` feature ([#23721](paradigmxyz/reth#23721)) - Add platform-specific RUSTFLAGS to Dockerfile ([#23738](paradigmxyz/reth#23738)) **Bench** - Add reorg mode to `new-payload-fcu` ([#23666](paradigmxyz/reth#23666)) - Enable `keccak-cache-global` in `reth-bb` ([#23723](paradigmxyz/reth#23723)) and fix multi-executor support ([#23763](paradigmxyz/reth#23763)) - Dedupe merged BAL storage reads ([#23758](paradigmxyz/reth#23758)) **Re-execute** - Verify reverts against changesets ([#23717](paradigmxyz/reth#23717)) **Payload** - Track Amsterdam block gas in builders ([#23743](paradigmxyz/reth#23743)) **CLI** - Use `TxTy`/`ReceiptTy` for static-file db get ([#23692](paradigmxyz/reth#23692)) and node types in execution stage dump ([#23705](paradigmxyz/reth#23705)) - Preserve `trusted_nodes_only` from config when `--trusted-only` is unset ([#23703](paradigmxyz/reth#23703)) - Avoid u64 underflow in `setup_without_evm` for genesis header ([#23728](paradigmxyz/reth#23728)) **Testing** - Add BAL request e2e coverage ([#23727](paradigmxyz/reth#23727)) - Cover admin node info discv5 port ([#23781](paradigmxyz/reth#23781)) **Misc** - Bump reth core to v0.3.1 ([#23707](paradigmxyz/reth#23707)) and alloy-evm to 0.33.3 ([#23778](paradigmxyz/reth#23778)) - Align ERA1 export with spec ([#23693](paradigmxyz/reth#23693)) ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-019dd89c-63d3-752f-ac22-7d4aef2ac3e3 - **Bumped reth git rev** from `7839f3d` to `73ec2c9` across all `reth-*` workspace dependencies in [Cargo.toml](file:///home/runner/work/tempo/tempo/Cargo.toml), and bumped `reth-primitives-traits` from `0.3.0` to `0.3.1` and `alloy-evm` from `0.33.0` to `0.34.0` to track upstream releases. - **Switched p2p proxy transactions channel** from `tokio::sync::mpsc::unbounded_channel` to `reth_metrics::common::mpsc::memory_bounded_channel` with a 32 MiB budget in [bin/tempo/src/p2p_proxy.rs](file:///home/runner/work/tempo/tempo/bin/tempo/src/p2p_proxy.rs), matching reth's new default `tx_channel_memory_limit_bytes` to prevent unbounded memory growth; added `reth-metrics` to [bin/tempo/Cargo.toml](file:///home/runner/work/tempo/tempo/bin/tempo/Cargo.toml). - **Changed `BlockExecutor::commit_transaction` signature** in [crates/evm/src/block.rs](file:///home/runner/work/tempo/tempo/crates/evm/src/block.rs) from returning `Result<GasOutput, BlockExecutionError>` to returning `GasOutput` directly (upstream removed the fallible return); replaced the missing-tx error with an `expect`, and updated all tests to drop `.unwrap()`. - **Added `Send + 'static` bound to `TxResult` impl** for `TempoTxResult<H>` to satisfy the new upstream trait bounds on associated execution-result types. - **Exposed `TempoBlockExecutor` and `TempoTxResult` as `pub`** (previously `pub(crate)`) and re-exported them from [crates/evm/src/lib.rs](file:///home/runner/work/tempo/tempo/crates/evm/src/lib.rs) so they can be named in the new associated types. - **Added associated types `TxExecutionResult` and `Executor` to `BlockExecutorFactory` impl** for `TempoEvmConfig`, and changed `create_executor` to return the concrete `Self::Executor<'a, DB, I>` instead of `impl BlockExecutorFor<'a, Self, DB, I>`, tracking upstream's move from RPIT to named associated types; dropped the now-unused `BlockExecutorFor` import and the `'a` lifetime bounds on `DB`/`I`. - **Implemented new `HeaderMut` methods** (`set_mix_hash`, `set_extra_data`, `set_parent_beacon_block_root`) for `TempoHeader` in [crates/primitives/src/reth_compat/header.rs](file:///home/runner/work/tempo/tempo/crates/primitives/src/reth_compat/header.rs) to satisfy the expanded upstream trait, delegating to the inner Ethereum header. [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/25101489247) --------- Co-authored-by: Centaur AI <ai@centaur.local> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: klkvr <klkvrr@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plumbs
ChangesetCachethrough historical state provider construction and extracts a cloneableOverlayBuilderso overlay configuration can be shared separately from the cachedOverlayStateProviderFactory.Historical state-root, storage-root, and proof paths now build their overlays through that shared builder while preserving the existing historical block semantics, and the engine/tree and parallel trie callers are updated to rebuild overlay factories from the provider factory plus builder when they need extended overlays.