perf(storage): batch trie updates across blocks in save_blocks#21140
Closed
perf(storage): batch trie updates across blocks in save_blocks#21140
Conversation
Based on #eng-perf Slack discussions identifying key bottlenecks: - update_history_indices: 26% of persist time - write_trie_updates: 25.4% - write_trie_changesets: 24.2% - Execution cache contention under high throughput New benchmarks: - execution_cache: cache hit rates, contention, TIP-20 patterns - heavy_persistence: accumulated blocks, history indices, state root - heavy_root: parallel vs sync at scale, large storage tries Includes runner script and optimization opportunities doc.
Previously, `update_history_indices` scanned the AccountChangeSets and StorageChangeSets database tables to build history indices after writing state. This was ~26% of persistence time per profiling in #eng-perf. This change collects account/storage transitions from the in-memory ExecutionOutcome during the block processing loop, avoiding the expensive DB cursor scans entirely. The history indices are then written directly from the pre-collected in-memory data. Optimization applies when: - save_mode.with_state() is true - History is not in RocksDB (MDBX path) Expected improvement: ~26% reduction in update_history_indices time.
Member
|
Closes #20611 |
mattsse
reviewed
Jan 30, 2026
Collaborator
mattsse
left a comment
There was a problem hiding this comment.
this now only has benches left
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Batches trie updates across all blocks in
save_blocksinstead of writing per-block.Problem
Per #eng-perf profiling,
write_trie_updateswas taking ~25% of persistence time. The current implementation callswrite_trie_updates_sortedonce per block, opening/closing cursors N times.In back-to-back (b2b) scenarios with 75-250 accumulated blocks, this overhead compounds significantly.
Solution
Accumulate trie updates across blocks using the existing
extend_refmethod, then write them all in a single batch:Expected Impact
write_trie_updatestime for b2b scenariosTesting
reth-providertests passRelated
georgios/history-indices-from-memory(26% history indices optimization)