Skip to content

perf: make Chain use DeferredTrieData#21137

Merged
mattsse merged 5 commits intomainfrom
dan/chain-deferred-1
Jan 17, 2026
Merged

perf: make Chain use DeferredTrieData#21137
mattsse merged 5 commits intomainfrom
dan/chain-deferred-1

Conversation

@Rjected
Copy link
Member

@Rjected Rjected commented Jan 16, 2026

This makes Chain use the DeferredTrieData instead of HashedPostStateSorted and TrieUpdatesSorted.

Key Changes

  • Deferred Trie Data: The Chain type now stores BTreeMap<BlockNumber, DeferredTrieData> instead of separate maps for trie updates and hashed state
  • Non-blocking Construction: Chain can be constructed without blocking on expensive trie computations
  • Lazy Access: Trie data is computed in background tasks and materialized only when needed

Accessing Trie Data

The trie data is still fully accessible, but access may block if background computation is in progress:

  • chain.trie_data() / chain.trie_data_at(num) - Returns DeferredTrieData handles (non-blocking)
  • chain.trie_updates() / chain.trie_updates_at(num) - Blocks until computation completes, then returns Arc<TrieUpdatesSorted>
  • chain.hashed_state() / chain.hashed_state_at(num) - Blocks until computation completes, then returns Arc<HashedPostStateSorted>

For best performance, prefer using trie_data() and calling wait_cloned() explicitly when you need the computed data.

Crate Structure

A new std-only crate reth-chain is added which contains:

  • Chain type
  • DeferredTrieData and ComputedTrieData
  • Bincode-compatible serde implementations for WAL persistence

Backwards Compatibility

WAL files created with older versions (without trie data) will deserialize correctly - missing fields default to empty maps.

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense

@@ -0,0 +1,69 @@
[package]
name = "reth-chain"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo this is fine

@github-project-automation github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Jan 16, 2026
@mattsse mattsse added the M-changelog This change should be included in the changelog label Jan 16, 2026
@Rjected Rjected force-pushed the dan/chain-deferred-1 branch 6 times, most recently from cde60a1 to cc5145a Compare January 16, 2026 19:21
@codspeed-hq
Copy link

codspeed-hq bot commented Jan 16, 2026

Merging this PR will not alter performance

✅ 118 untouched benchmarks
⏩ 7 skipped benchmarks1


Comparing dan/chain-deferred-1 (b4fd9dd) with main (b96a308)

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@mattsse mattsse force-pushed the dan/chain-deferred-1 branch from cc5145a to 95d5202 Compare January 16, 2026 22:52
@mattsse mattsse force-pushed the dan/chain-deferred-1 branch from 81ba6a8 to f910cce Compare January 16, 2026 23:23
@mattsse mattsse enabled auto-merge January 16, 2026 23:26
@mattsse mattsse force-pushed the dan/chain-deferred-1 branch from cc9c2b8 to f910cce Compare January 17, 2026 00:12
Use serde_with::SerializeAs directly instead of From impl for
ExExNotification serialization, matching how Chain handles its
DeferredTrieData (which requires collecting owned data before borrowing).
Restore reth-trie-common as unconditional dependency in execution-types
and remove #[cfg(feature = "std")] from HashedPostState imports and
hash_state_slow method.
@mattsse mattsse added this pull request to the merge queue Jan 17, 2026
Merged via the queue into main with commit c617d25 Jan 17, 2026
43 of 44 checks passed
@mattsse mattsse deleted the dan/chain-deferred-1 branch January 17, 2026 01:14
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Jan 17, 2026
mattsse added a commit that referenced this pull request Jan 17, 2026
Add no-std compatible lazy initialization wrapper for trie data:

- SortedTrieData: Container bundling HashedPostStateSorted and
  TrieUpdatesSorted together
- LazyTrieData: Lazily initialized wrapper supporting ready mode
  (immediate data) and deferred mode (computed on first access)

These types provide a clean abstraction for lazy trie data without
requiring the complexity of the reth-chain crate introduced in #21137.

Amp-Thread-ID: https://ampcode.com/threads/T-019bc9ce-d450-7307-bc10-04dd112904f0
Co-authored-by: Amp <amp@ampcode.com>
mattsse added a commit that referenced this pull request Jan 17, 2026
This PR reverts #21137 (the reth-chain crate) and adds the useful
LazyTrieData/SortedTrieData types directly to reth-trie-common.

Changes:
- Move Chain back to reth-execution-types
- Move DeferredTrieData back to reth-chain-state
- Remove the reth-chain crate
- Add LazyTrieData and SortedTrieData to reth-trie-common

The new types in reth-trie-common:
- SortedTrieData: Container bundling HashedPostStateSorted + TrieUpdatesSorted
- LazyTrieData: No-std compatible lazy wrapper with ready/deferred modes
mattsse added a commit that referenced this pull request Jan 17, 2026
This PR reverts #21137 (the reth-chain crate) and adds the useful
LazyTrieData/SortedTrieData types directly to reth-trie-common.

Changes:
- Move Chain back to reth-execution-types
- Move DeferredTrieData back to reth-chain-state
- Remove the reth-chain crate
- Add LazyTrieData and SortedTrieData to reth-trie-common

The new types in reth-trie-common:
- SortedTrieData: Container bundling HashedPostStateSorted + TrieUpdatesSorted
- LazyTrieData: No-std compatible lazy wrapper with ready/deferred modes
mattsse added a commit that referenced this pull request Jan 17, 2026
This PR reverts #21137 (the reth-chain crate) and adds the useful
LazyTrieData/SortedTrieData types directly to reth-trie-common.

Changes:
- Move Chain back to reth-execution-types
- Move DeferredTrieData back to reth-chain-state
- Remove the reth-chain crate
- Add LazyTrieData and SortedTrieData to reth-trie-common

The new types in reth-trie-common:
- SortedTrieData: Container bundling HashedPostStateSorted + TrieUpdatesSorted
- LazyTrieData: No-std compatible lazy wrapper with ready/deferred modes
mattsse added a commit that referenced this pull request Jan 17, 2026
This PR reverts #21137 (the reth-chain crate) and adds the useful
LazyTrieData/SortedTrieData types directly to reth-trie-common.

Changes:
- Move Chain back to reth-execution-types
- Move DeferredTrieData back to reth-chain-state
- Remove the reth-chain crate
- Add LazyTrieData and SortedTrieData to reth-trie-common

The new types in reth-trie-common:
- SortedTrieData: Container bundling HashedPostStateSorted + TrieUpdatesSorted
- LazyTrieData: No-std compatible lazy wrapper with ready/deferred modes
Vui-Chee added a commit to okx/reth that referenced this pull request Jan 20, 2026
* tag 'v1.10.1': (49 commits)
  chore: bump version to 1.10.1 (paradigmxyz#21188)
  chore: rename extend_ref methods on sorted data structures (paradigmxyz#21043)
  fix(flashblocks): Add flashblock ws connection retry period (paradigmxyz#20510)
  chore(bench): add --disable-tx-gossip to benchmark node args (paradigmxyz#21171)
  refactor(stages): reuse history index cache buffers in `collect_history_indices` (paradigmxyz#21017)
  feat(download): resumable snapshot downloads with auto-retry (paradigmxyz#21161)
  ci: update to tempoxyz (paradigmxyz#21176)
  chore: apply spelling and typo fixes (paradigmxyz#21182)
  docs: document minimal storage mode in pruning FAQ (paradigmxyz#21025)
  chore(deps): weekly `cargo update` (paradigmxyz#21167)
  feat(execution-types): add receipts_iter helper (paradigmxyz#21162)
  revert: undo Chain crate, add LazyTrieData to trie-common (paradigmxyz#21155)
  feat(engine): add new_payload_interval metric (start-to-start) (paradigmxyz#21159)
  feat(engine): add time_between_new_payloads metric (paradigmxyz#21158)
  fix(storage-api): gate reth-chain dependency behind std feature
  perf(storage): batch trie updates across blocks in save_blocks (paradigmxyz#21142)
  refactor: use ExecutionOutcome::single instead of tuple From (paradigmxyz#21152)
  chore(chain-state): reorganize deferred_trie.rs impl blocks (paradigmxyz#21151)
  feat(primitives-traits): add try_recover_signers for parallel batch recovery (paradigmxyz#21103)
  perf: make Chain use DeferredTrieData (paradigmxyz#21137)
  ...
theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request Jan 22, 2026
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request Feb 11, 2026
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

M-changelog This change should be included in the changelog

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants