feat: parallelize `save_blocks` #20993

Merged

joshieDo merged 13 commits intomainfrom

joshie/par-save-blocks

Jan 15, 2026

Collaborator

joshieDo commented Jan 13, 2026 •

edited

Loading

parallelize save_blocks
insert_block now calls save_blocks

PR Stack

#21045 (metrics)    #21003 (rocksdb)
         \           /
          \         /
           #21012 (account changesets)
              ↓
           #20993 (parallelize save_blocks) ◀
              ↓
            main

joshieDo self-assigned this

joshieDo added A-db A-static-files labels

github-project-automation bot added this to Reth Tracker

github-project-automation bot moved this to Backlog in Reth Tracker

joshieDo changed the base branch from main to joshie/split-commit-sf

January 13, 2026 13:03

joshieDo force-pushed the joshie/split-commit-sf branch from 9bd7808 to 4629f21 Compare

January 13, 2026 14:57

joshieDo force-pushed the joshie/par-save-blocks branch 2 times, most recently from 01196a8 to f101e09 Compare

January 13, 2026 15:48

joshieDo force-pushed the joshie/split-commit-sf branch from 4629f21 to 6275ac6 Compare

January 13, 2026 15:48

joshieDo force-pushed the joshie/par-save-blocks branch from f101e09 to 868f884 Compare

January 13, 2026 15:58

mattsse requested changes

View reviewed changes

Collaborator

mattsse left a comment

some suggestions, but afaict the flow makes sense, in parallelizes as much as possible

crates/optimism/cli/src/commands/import_receipts.rs Outdated

                       // finally, write the receipts
-                      provider.write_state(&execution_outcome, OriginalValuesKnown::Yes)?;
+                      provider.write_state(&execution_outcome, OriginalValuesKnown::Yes, true)?;

Collaborator

mattsse Jan 13, 2026

can we make this a config struct because i assume well add more settings eventually?

Collaborator Author

joshieDo Jan 13, 2026

#21012 added here

crates/storage/provider/src/providers/database/provider.rs

Comment on lines -3039 to -3059

-                  /// Inserts the block into the database, always modifying the following static file segments and
-                  /// tables:
-                  /// * [`StaticFileSegment::Headers`]
-                  /// * [`tables::HeaderNumbers`]
-                  /// * [`tables::BlockBodyIndices`]
-                  ///
-                  /// If there are transactions in the block, the following static file segments and tables will
-                  /// be modified:
-                  /// * [`StaticFileSegment::Transactions`]
-                  /// * [`tables::TransactionBlocks`]
+                  /// Inserts the block into the database, writing to both static files and MDBX.
                   ///
-                  /// If ommers are not empty, this will modify [`BlockOmmers`](tables::BlockOmmers).
-                  /// If withdrawals are not empty, this will modify
-                  /// [`BlockWithdrawals`](tables::BlockWithdrawals).
-                  ///
-                  /// If the provider has __not__ configured full sender pruning, this will modify either:
-                  /// * [`StaticFileSegment::TransactionSenders`] if senders are written to static files
-                  /// * [`tables::TransactionSenders`] if senders are written to the database
-                  ///
-                  /// If the provider has __not__ configured full transaction lookup pruning, this will modify
-                  /// [`TransactionHashNumbers`](tables::TransactionHashNumbers).

Collaborator

mattsse Jan 13, 2026

is this no longer accurate?

crates/storage/provider/src/providers/database/provider.rs Outdated Show resolved Hide resolved

crates/storage/provider/src/providers/database/provider.rs

-                  /// [`TransactionHashNumbers`](tables::TransactionHashNumbers).
+                  /// This is a convenience method primarily used in tests. For production use,
+                  /// prefer [`Self::save_blocks`] which handles execution output and trie data.
                   fn insert_block(

Collaborator

mattsse Jan 13, 2026

when do we even call this?

separately

reth/crates/storage/provider/src/providers/database/provider.rs

Line 3369 in f101e09

fn append_blocks_with_state(

appears to be unused?

crates/storage/provider/src/providers/database/provider.rs

-                  ///
-                  /// If the provider has __not__ configured full transaction lookup pruning, this will modify
-                  /// [`TransactionHashNumbers`](tables::TransactionHashNumbers).
+                  /// This is a convenience method primarily used in tests. For production use,

Collaborator

mattsse Jan 13, 2026

ah nvm, I think I understand now

github-project-automation bot moved this from Backlog to In Progress in Reth Tracker

joshieDo mentioned this pull request

feat: add rocksdb to save_blocks #21003

Merged

joshieDo force-pushed the joshie/par-save-blocks branch 2 times, most recently from b271a28 to 4fdb472 Compare

January 13, 2026 16:25

joshieDo marked this pull request as ready for review

January 13, 2026 16:28

joshieDo requested review from Rjected, fgimenez, gakonst, mediocregopher, rakita, shekhirin and yongkangc as code owners

January 13, 2026 16:28

joshieDo force-pushed the joshie/par-save-blocks branch 2 times, most recently from 8f2cbc4 to 76c93c6 Compare

January 13, 2026 16:30

Base automatically changed from joshie/split-commit-sf to main

January 13, 2026 16:38

joshieDo force-pushed the joshie/par-save-blocks branch 2 times, most recently from 705619b to c38dbb8 Compare

January 13, 2026 16:47


          parallelize save_blocks

cf89d57

joshieDo marked this pull request as draft

January 13, 2026 18:18

joshieDo mentioned this pull request

feat: enable account changesets on save_blocks #21012

Merged

joshieDo marked this pull request as ready for review

January 13, 2026 18:35

joshieDo requested a review from mattsse

January 13, 2026 18:44

pepyakin approved these changes

View reviewed changes

Member

pepyakin left a comment

I've went through the code and it looks reasonable to me.

yongkangc reviewed

View reviewed changes

crates/storage/provider/src/providers/database/provider.rs

+                          for (i, block) in blocks.iter().enumerate() {
+                              let recovered_block = block.recovered_block();
+                              let start = Instant::now();

Member

yongkangc Jan 14, 2026

should we have the instant::now() outside the loop? right now in this scope we have a couple of this ~5

Collaborator

mattsse Jan 14, 2026

this seems fine, I think we want this per block

yongkangc reviewed

View reviewed changes

crates/storage/provider/src/providers/database/provider.rs Outdated Show resolved Hide resolved

yongkangc approved these changes

View reviewed changes

Member

yongkangc left a comment

@joshieDo left some comments, overall flow lgtm

joshieDo mentioned this pull request

feat: update metrics on save_blocks #21045

Merged

mattsse approved these changes

View reviewed changes

Collaborator

mattsse left a comment

checks out

crates/storage/provider/src/providers/database/provider.rs

+                          for (i, block) in blocks.iter().enumerate() {
+                              let recovered_block = block.recovered_block();
+                              let start = Instant::now();

Collaborator

mattsse Jan 14, 2026

this seems fine, I think we want this per block

joshieDo and others added 4 commits

January 14, 2026 17:39


          feat: enable account changesets on save_blocks (#21012)

fadc97a


          Merge branch 'main' into joshie/par-save-blocks

c2ddcb2


          feat(storage): add tracing spans (#21072)

84bdf71

Co-authored-by: Sergei Shulepov <pep@tempo.xyz>


          Merge branch 'main' into joshie/par-save-blocks

75eb667

mediocregopher mentioned this pull request

perf(db): Sort transaction hash/nums before inserting them #21096

Closed

joshieDo added this pull request to the merge queue

github-merge-queue bot removed this pull request from the merge queue due to failed status checks

mediocregopher and others added 5 commits

January 15, 2026 14:03


          perf(db): insert transaction hash numbers presorted by tx hash

c2323cc


          Revert "perf(db): insert transaction hash numbers presorted by tx hash"

c94309e

This reverts commit c2323cc.


          Reapply "perf(db): insert transaction hash numbers presorted by tx hash"

56e5c01

This reverts commit c94309e.


          Merge branch 'main' into joshie/par-save-blocks

9f13bac


          dupped import

6036b41

joshieDo enabled auto-merge

January 15, 2026 14:23

joshieDo added this pull request to the merge queue

Merged via the queue into main with commit f012b33

44 checks passed

joshieDo deleted the joshie/par-save-blocks branch

January 15, 2026 15:17

github-project-automation bot moved this from In Progress to Done in Reth Tracker

theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request


          feat: parallelize save_blocks (paradigmxyz/reth#20993)

Co-authored-by: Sergei Shulepov <s.pepyakin@gmail.com>
Co-authored-by: Sergei Shulepov <pep@tempo.xyz>
Co-authored-by: Brian Picciano <me@mediocregopher.com>

theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request


          feat: parallelize save_blocks (paradigmxyz/reth#20993)

3bd1ce1

Co-authored-by: Sergei Shulepov <s.pepyakin@gmail.com>
Co-authored-by: Sergei Shulepov <pep@tempo.xyz>
Co-authored-by: Brian Picciano <me@mediocregopher.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

pepyakin pepyakin approved these changes

mattsse mattsse approved these changes

yongkangc yongkangc approved these changes

rakita Awaiting requested review from rakita

shekhirin Awaiting requested review from shekhirin shekhirin is a code owner

Rjected Awaiting requested review from Rjected Rjected is a code owner

mediocregopher Awaiting requested review from mediocregopher mediocregopher is a code owner

fgimenez Awaiting requested review from fgimenez

gakonst Awaiting requested review from gakonst gakonst is a code owner

Labels

A-db A-static-files