Skip to content

perf(storage): batch trie updates across blocks in save_blocks#21140

Closed
gakonst wants to merge 7 commits intomainfrom
georgios/batch-trie-updates
Closed

perf(storage): batch trie updates across blocks in save_blocks#21140
gakonst wants to merge 7 commits intomainfrom
georgios/batch-trie-updates

Conversation

@gakonst
Copy link
Member

@gakonst gakonst commented Jan 16, 2026

Summary

Batches trie updates across all blocks in save_blocks instead of writing per-block.

Problem

Per #eng-perf profiling, write_trie_updates was taking ~25% of persistence time. The current implementation calls write_trie_updates_sorted once per block, opening/closing cursors N times.

In back-to-back (b2b) scenarios with 75-250 accumulated blocks, this overhead compounds significantly.

Solution

Accumulate trie updates across blocks using the existing extend_ref method, then write them all in a single batch:

// Accumulate across blocks
let mut accumulated_trie_updates: Option<TrieUpdatesSorted> = None;

for block in blocks {
    // ... other per-block writes ...
    
    match &mut accumulated_trie_updates {
        Some(acc) => acc.extend_ref(&trie_data.trie_updates),
        None => accumulated_trie_updates = Some((*trie_data.trie_updates).clone()),
    }
}

// Single batch write at end
if let Some(trie_updates) = &accumulated_trie_updates {
    self.write_trie_updates_sorted(trie_updates)?;
}

Expected Impact

  • ~50% reduction in write_trie_updates time for b2b scenarios
  • Reduces cursor open/close overhead from N to 1
  • Reduces MDBX transaction overhead

Testing

  • All existing reth-provider tests pass

Related

Based on #eng-perf Slack discussions identifying key bottlenecks:
- update_history_indices: 26% of persist time
- write_trie_updates: 25.4%
- write_trie_changesets: 24.2%
- Execution cache contention under high throughput

New benchmarks:
- execution_cache: cache hit rates, contention, TIP-20 patterns
- heavy_persistence: accumulated blocks, history indices, state root
- heavy_root: parallel vs sync at scale, large storage tries

Includes runner script and optimization opportunities doc.
Previously, `update_history_indices` scanned the AccountChangeSets and
StorageChangeSets database tables to build history indices after writing
state. This was ~26% of persistence time per profiling in #eng-perf.

This change collects account/storage transitions from the in-memory
ExecutionOutcome during the block processing loop, avoiding the expensive
DB cursor scans entirely. The history indices are then written directly
from the pre-collected in-memory data.

Optimization applies when:
- save_mode.with_state() is true
- History is not in RocksDB (MDBX path)

Expected improvement: ~26% reduction in update_history_indices time.
@mediocregopher
Copy link
Member

Closes #20611

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this now only has benches left

@mattsse mattsse closed this Jan 30, 2026
@github-project-automation github-project-automation bot moved this from Backlog to Done in Reth Tracker Jan 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants