perf(trie): Use ParallelSparseTrie (if enabled) for storage tries by mediocregopher · Pull Request #17959 · paradigmxyz/reth

mediocregopher · 2025-08-20T12:12:36Z

No description provided.

Rjected

ah this is a nice way to enable the parallel sparse trie for storage tries with the flag

Rjected · 2025-08-20T14:50:53Z

crates/trie/sparse/src/state.rs

+    /// A default cleared trie instance, which will be cloned when creating new tries.
+    default_trie: SparseTrie<S>,


ah nice, this makes sense

Rjected · 2025-08-20T14:53:00Z

should only merge once we decide we want this enabled w the flag

mediocregopher · 2025-08-22T16:00:34Z

In benchmarking this we found that it doesn't seem to provide any significant benefit over SerialSparseTrie for either base or mainnet. One note was that within SparseStateTrie::update_account we call root on the storage trie, which should be very fast because it was already computed in a previous step, but this takes significantly longer with ParallelSparseTrie vs SerialSparseTrie.

We should address this and then re-attempt benchmarks.

fynnss · 2025-08-25T12:46:31Z

crates/trie/sparse-parallel/src/trie.rs

    ) -> (Vec<ChangedSubtrie>, PrefixSetMut) {
+        // Fast-path: If the prefix set is empty then no subtries can have been changed. Just return
+        // empty values.
+        if prefix_set.is_empty() {


I think it's not worth parallelizing computation when the number of keys is less than 256; it actually incurs the overhead of asynchronous concurrency.

Right, I think the next thing which will be worth investigating here is some kind threshold within ParallelSparseTrie under which it does everything serially. But that will be a larger effort, likely requiring some independent PRs prior to this one.

In go-ethereum, the hasher only performs 16-way parallel computation of hashes when the number of unhashed nodes exceeds 100, which can serve as a reference.

https://github.com/ethereum/go-ethereum/blob/master/trie/trie.go#L741-L752

bnb-chain/reth@a21b98b I add a dynamic sparse trie. Please help review if you're interested.

Have you been able to benchmark this implementation? I would expect that this TODO would need to be addressed before it becomes worthwhile; the overhead of rehashing all the node which were previously revealed would overshadow any benefit I'd guess, but could be wrong.

yes, the overhead of rehashing all the node is huge. I'll test your implementation on bsc mainnet first.

…storage-tries

gakonst · 2025-08-28T12:44:10Z

Should the parallel threshold be on a per account basis? Ie some accounts on a block might have lots of storage updates, which means the PST might be worth it, vs the accounts with few updates serial is better?

mediocregopher · 2025-08-28T14:21:02Z

Should the parallel threshold be on a per account basis? Ie some accounts on a block might have lots of storage updates, which means the PST might be worth it, vs the accounts with few updates serial is better?

This is how it's implemented, we have one instance which handles the accounts trie, and then one instance per-storage trie. Each trie checks its own contents to decide whether to be parallel or not, so an account with lots of updates to its storage will hit the threshold but another with only a couple won't

mediocregopher · 2025-08-28T14:27:33Z

I've introduced a --engine.parallel-sparse-trie-threshold CLI arg which defaults to 100. If the number of updates to a trie does not exceed that threshold then it will do root computation serially. Similarly when revealing nodes if the number of nodes being revealed does not exceed the threshold then revealing is done serially.

Initial results are encouraging, getting a consistent 5% improvement on mainnet, haven't gotten to test base yet.

I'm going to play around with different threshold values and see if it can be improved further.

Rjected

makes sense, so this basically controls whether or not we use rayon depending on the number of parallel tasks we would have

crates/engine/primitives/src/config.rs

crates/engine/tree/src/tree/payload_processor/mod.rs

crates/node/core/src/node_config.rs

crates/trie/sparse-parallel/src/trie.rs

fynnss · 2025-08-29T08:49:52Z

@mediocregopher I've tested this feature. But it occurs root hash mismatch on bsc mainnet.

2025-08-29T06:43:18.925299Z  INFO Block added to canonical chain number=59268694 hash=0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74 peers=9 txs=163 gas_used=22.61Mgas gas_throughput=575.67Mgas/second gas_limit=75.00Mgas full=30.1% base_fee=0.00Gwei blobs=1 excess_blobs=0 elapsed=39.271122ms
2025-08-29T06:43:18.933959Z  INFO Canonical chain committed number=59268694 hash=0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74 elapsed=283.737µs
2025-08-29T06:43:18.933976Z  INFO Forkchoice updated head_block_hash=0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74 safe_block_hash=0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74 finalized_block_hash=0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74
2025-08-29T06:43:21.262714Z  INFO Received block from consensus engine number=59268697 hash=0xaccb6b0ad83efd7225d4f9815d43bc8508af72826beccd47a08f4deaa97d0312
2025-08-29T06:43:21.263758Z  INFO Received forkchoice updated message when syncing head_block_hash=0xaccb6b0ad83efd7225d4f9815d43bc8508af72826beccd47a08f4deaa97d0312 safe_block_hash=0xaccb6b0ad83efd7225d4f9815d43bc8508af72826beccd47a08f4deaa97d0312 finalized_block_hash=0xaccb6b0ad83efd7225d4f9815d43bc8508af72826beccd47a08f4deaa97d0312
2025-08-29T06:43:21.301780Z  INFO Received block from consensus engine number=59268697 hash=0xaccb6b0ad83efd7225d4f9815d43bc8508af72826beccd47a08f4deaa97d0312
2025-08-29T06:43:22.116526Z  INFO State root task finished state_root=0x536c4b469be570aa72178ac6c64a0229809aa9f56ec5513a5e4569c78d6a7847 elapsed=2.91215ms
2025-08-29T06:43:22.116551Z  WARN State root task returned incorrect state root state_root=0x536c4b469be570aa72178ac6c64a0229809aa9f56ec5513a5e4569c78d6a7847 block_state_root=0x243650b9d85512646b44c9c327ab6a509221a552c59270ffb2a26476e5f999ba
2025-08-29T06:43:22.118046Z  WARN Failed to compute state root in parallel block=NumHash { number: 59268695, hash: 0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc } persisting_kind=NotPersisting
2025-08-29T06:43:22.497389Z  WARN State root mismatch after re-execution original_root=0xb8e9001d2086aed40525497aa1338e78c9cabc80d3ec2a052531a83b6d38af17 re_executed_root=0x34ea5e50f23cb8e29add8bd4f75a6adcae81a7f1d26dda1066bdd1b5b78f3214 diff_path=./data_dir/invalid_block_hooks/witness/59268695_0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc.state_root.diff
2025-08-29T06:43:22.497466Z  WARN Re-executed state root does not match block state root header_state_root=0x243650b9d85512646b44c9c327ab6a509221a552c59270ffb2a26476e5f999ba re_executed_root=0x34ea5e50f23cb8e29add8bd4f75a6adcae81a7f1d26dda1066bdd1b5b78f3214 diff_path=./data_dir/invalid_block_hooks/witness/59268695_0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc.header_state_root.diff
2025-08-29T06:43:22.503185Z  WARN Failed to invoke hook err=key must be a string
2025-08-29T06:43:22.503907Z  WARN Invalid block error on new payload invalid_hash=0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc invalid_number=59268695 validation_err=mismatched block state root: got 0xb8e9001d2086aed40525497aa1338e78c9cabc80d3ec2a052531a83b6d38af17, expected 0x243650b9d85512646b44c9c327ab6a509221a552c59270ffb2a26476e5f999ba
2025-08-29T06:43:22.503991Z  WARN Bad block with hash invalid_ancestor=BlockWithParent { parent: 0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74, block: NumHash { number: 59268695, hash: 0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc } }
2025-08-29T06:43:22.504092Z  WARN Encountered invalid block number=59268695 hash=0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc
2025-08-29T06:43:23.124704Z  INFO Received block from consensus engine number=59268698 hash=0x0eb501fce46d42558c5cfbc9fbc7ad7506b21e328867904e115726cc19109d16
2025-08-29T06:43:23.126000Z  INFO Received invalid forkchoice updated message head_block_hash=0x0eb501fce46d42558c5cfbc9fbc7ad7506b21e328867904e115726cc19109d16 safe_block_hash=0x0eb501fce46d42558c5cfbc9fbc7ad7506b21e328867904e115726cc19109d16 finalized_block_hash=0x0eb501fce46d42558c5cfbc9fbc7ad7506b21e328867904e115726cc19109d16
2025-08-29T06:43:23.411428Z  INFO Received block from consensus engine number=59268700 hash=0x50ee6ee3f8607f512526009f7165f1e7cd3765966b2ed7dcb9178cce0249c153
2025-08-29T06:43:23.412413Z  INFO Received forkchoice updated message when syncing head_block_hash=0x50ee6ee3f8607f512526009f7165f1e7cd3765966b2ed7dcb9178cce0249c153 safe_block_hash=0x50ee6ee3f8607f512526009f7165f1e7cd3765966b2ed7dcb9178cce0249c153 finalized_block_hash=0x50ee6ee3f8607f512526009f7165f1e7cd3765966b2ed7dcb9178cce0249c153
2025-08-29T06:43:23.853403Z  WARN Encountered invalid block number=59268695 hash=0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc
2025-08-29T06:43:26.514330Z  INFO Received block from consensus engine number=59268704 hash=0x508fefd286ec18f62cf8ff27216f2209b04463e9f0c395da03b954a055fc0f0f
2025-08-29T06:43:26.515527Z  INFO Received forkchoice updated message when syncing head_block_hash=0x508fefd286ec18f62cf8ff27216f2209b04463e9f0c395da03b954a055fc0f0f safe_block_hash=0x508fefd286ec18f62cf8ff27216f2209b04463e9f0c395da03b954a055fc0f0f finalized_block_hash=0x508fefd286ec18f62cf8ff27216f2209b04463e9f0c395da03b954a055fc0f0f
2025-08-29T06:43:27.078646Z  INFO Received block from consensus engine number=59268705 hash=0x72f35ed5907f0438219ed60ecd3e35c55bd75769c920a1d380a0a300e225a256
2025-08-29T06:43:27.079576Z  INFO Received forkchoice updated message when syncing head_block_hash=0x72f35ed5907f0438219ed60ecd3e35c55bd75769c920a1d380a0a300e225a256 safe_block_hash=0x72f35ed5907f0438219ed60ecd3e35c55bd75769c920a1d380a0a300e225a256 finalized_block_hash=0x72f35ed5907f0438219ed60ecd3e35c55bd75769c920a1d380a0a300e225a256
2025-08-29T06:43:27.234323Z  WARN Bad block with existing invalid ancestor hash=0x098eb64d7d159b8baae4e3b2db1e496fbbb6045cd8b06a3e805e4cd3d4861f0f invalid_ancestor=BlockWithParent { parent: 0xc90f75b12bac4092f27ddeecd56086b402e10d018bc1b6d9866406adc7de9d74, block: NumHash { number: 59268695, hash: 0x792cbe8b8be561c3c7bc3b2be2316210b409b2f448ea0c6978f08d1d005534dc } }

And then it can not auto recovery. The data is corrupted.

Invalid state root error on stage verification!
This is an error that likely requires a report to the reth team with additional information.
Please include the following information in your report:
 * This error message
 * The state root of the block that was rejected
 * The output of `reth db stats --checksum` from the database that was being used. This will take a long time to run!
 * 50-100 lines of logs before and after the first occurrence of the log message with the state root of the block that was rejected.
 * The debug logs from __the same time period__. To find the default location for these logs, run:
   `reth --help | grep -A 4 'log.file.directory'`

Once you have this information, please submit a github issue at https://github.com/paradigmxyz/reth/issues/new
 target_block=59268729 got=0x7025d88b3d701512cf7644af05a1709ef6ef12f49560c3a73354ea4cc1b2fd21 expected=SealedHeader { hash: OnceLock(0xf275b30966ea5dd4c05353229c4045e798f5d1b1b273c359c1acf506f4aa4671), header: Header { parent_hash: 0xd43ea5bd7efa01e3d0d73831aff3afd500a8c0b60b745ed25132bfc052182600, ommers_hash: 0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347, beneficiary: 0x0dc5e1cae4d364d0c79c9ae6bddb5da49b10a7d9, state_root: 0xfd81881eca1ce4faa2151b1cba8a2fa70c349e8a0f48ff67dc4e6f3f60a11806, transactions_root: 0x3d2cf6b629d042b7908d2d763075f282877cac38c68390d22b48bde23c5d64ce, receipts_root: 0x10c8454cbaaaf1db24efb7d1a37b634a153b198e4127538ad30e80e5bcb01eca, logs_bloom: 0x1df786430f227b0cab304ce5da5358289d50fa171491129475e8623147e3ffe19e05f03297449d0ca309d1277cf3829941218ab4170a3b1156a25e4521a48cbe4cdcb6174745e039a3d2e15c89300370a6790837f17c802d4655620985e14803c64e02ec47ea45c49c348cbf8749f85fc991a66dc428c454986bf055e6a876f79780706dbf1b4b59c2ad25e5dc3045a7577c55a703142a0c0e89e3d861836e61b63d34eec178253d2f476dd18371b81563b18ac25bda3848932a27b07125a57702b6758e844ad42c88c44a01a05e871274359f1b92a40371f7dd75260572faf88d700b13129af2ae4d69278603ea07733c50a0234b906649e408caf6b2b0e86a, difficulty: 2, number: 59268729, gas_limit: 75000000, gas_used: 17698628, timestamp: 1756449825, extra_data: 0xd983010511846765746889676f312e32332e3130856c696e75780000098d24acf8b5831effffb8608bde49700340337565938b7029f9231167270cf377f72b9a829dd1b565cf8786e021cb36d77761ac7c6065854d08cfa409acca049b5998d46d6f114435444fe7f015fc9dff9a0ea3665ee5cc53f99099b8d757de8c61040928442aeacf71df3af84c8403885e77a02da51de96b889b00127db7f5fb1d263c763ae4330d19c7998778b23edb3048538403885e78a0d43ea5bd7efa01e3d0d73831aff3afd500a8c0b60b745ed25132bfc052182600801ccc410db5de839970a04c99c965f45da210fef2f31f3b121ec4d28a82f9f3070b0a5dd431525f5a8b7888eecbdb89baa8bdde35a8b3ef4211f2411a5cc6c43c00, mix_hash: 0x0000000000000000000000000000000000000000000000000000000000000000, nonce: 0x0000000000000000, base_fee_per_gas: Some(0), withdrawals_root: Some(0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421), blob_gas_used: Some(131072), excess_blob_gas: Some(0), parent_beacon_block_root: Some(0x0000000000000000000000000000000000000000000000000000000000000000), requests_hash: Some(0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855) } }
2025-08-29T06:43:53.020619Z ERROR Stage encountered a validation error: mismatched block state root: got 0x7025d88b3d701512cf7644af05a1709ef6ef12f49560c3a73354ea4cc1b2fd21, expected 0xfd81881eca1ce4faa2151b1cba8a2fa70c349e8a0f48ff67dc4e6f3f60a11806 stage=MerkleExecute bad_block=59268729

Do you know any way to recover it?

mediocregopher · 2025-08-29T08:59:25Z

@fynnss You can re-generate the trie tables by doing:

reth stage drop --datadir <datadir> --chain <chain> merkle

The next time you start reth node it will regenerate the trie tables, which could take some hours depending on the chain size.

Can you link a branch that you're working off of? Specifically I want to know if you have included commit 4fe6ae4, which fixes state root mismatches we ran into when enabling PST for storage tries.

fynnss · 2025-08-29T09:05:24Z

Thank you so much. I've checked and it doesn't include this commit [4fe6ae4], next I'll cherry-pick this commit and retest.

This reverts commit db6c0f2.

…storage-tries

mediocregopher · 2025-09-01T10:56:08Z

@Rjected for now I've removed the CLI argument, it's not clear why a user would ever use it over just disabling the PST, and trying to document how to use it would just cause more confusion than anything.

I've also split the threshold value into a struct with two fields, mostly to make the code clearer.

…storage-tries

mattsse

lgtm, pending @Rjected

* feat: optionally disable balance check for `EthTransactionValidator` (paradigmxyz#18086) * feat(optimism): Launch `FlashBlockService` when websocket URL is provided in `OpEthApi` (paradigmxyz#18077) * feat: add NoopNetwork example (paradigmxyz#18093) * feat: add helper for provider with wallet (paradigmxyz#18085) * feat(optimism): Add `flashblocks_url` as part of rollup args of the `op-reth` CLI (paradigmxyz#18094) * perf(engine): only fetch headers instead of full blocks for tree tasks (paradigmxyz#18088) * feat(optimism): Implement conversion of `ExecutionPayloadBaseV1` into `OpNextBlockEnvAttributes` (paradigmxyz#18097) * feat(optimism): Remove builder of next block environment from `FlashBlockService` (paradigmxyz#18100) * refactor: make transaction validator functions reusable (paradigmxyz#17929) Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com> * feat: add module manipulation methods and RPC server arg helpers (paradigmxyz#18084) * chore(reth-optimism-cli): use OpTypedTransaction::eip2718_encode (paradigmxyz#18105) * ci: remove expected failures (paradigmxyz#18099) * feat: add EIP-7934 block size check to validateBuilderSubmissionV5 (paradigmxyz#18111) * chore: include err in log (paradigmxyz#18119) * fix(optimism): Prevent old pending flashblock from being returned from `pending_flashblock` (paradigmxyz#18103) * chore: make `caller_gas_allowance` an RPC trait method (paradigmxyz#18101) * fix(optimism): Fix endless poll on the FlashBlockService (paradigmxyz#18120) * chore: add prewarm traces (paradigmxyz#18117) * refactor(eth-wire): remove EthVersion::total_messages in favor of EthMessageID::max (paradigmxyz#17999) * fix(engine): Prevent instant miner from creating empty blocks (paradigmxyz#18108) Signed-off-by: 7suyash7 <suyashnyn1@gmail.com> * perf(engine): only clone headers instead of full blocks for tree tasks (paradigmxyz#18116) * fix(optimism): Verify that flashblocks are not old according to canon state (paradigmxyz#18123) * fix: import should count on the delta (paradigmxyz#17819) Signed-off-by: tmel <tmel0103@gmail.com> Signed-off-by: tmelhao <tmel0103@gmail.com> Co-authored-by: tmel <tmel0103@gmail.com> * feat: Forward transactions to a specified endpoint (paradigmxyz#17444) Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com> * feat(net): implement support of subprotocols (paradigmxyz#18080) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fix(optimism): Fail if latest and base flashblock parent are different (paradigmxyz#18132) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fix(txpool): ensure fee changes are updated (paradigmxyz#18137) * refactor: merge `EthTransactionValidator` and `EthTransactionValidatorInner` (paradigmxyz#18129) * docs(op): op chains don't require deposit contracts, so as dev chain (paradigmxyz#17988) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * feat: generalize impl EngineValidatorAddOn for OpAddOns (paradigmxyz#18141) * perf(engine): pre-allocate Vec capacity in payload processor (paradigmxyz#18148) * perf(engine): pre-allocate channel handles in prewarm task (paradigmxyz#18147) * chore(reth-optimism-storage): small refactor code (paradigmxyz#18104) * fix(trie): Fix call to update_account in witness (paradigmxyz#18154) * feat(optimism): add FlashblocksRx getter (paradigmxyz#18155) * perf(reth-invalid-block-hooks): use Reverts::eq reduce clone (paradigmxyz#18159) * perf(optimism): Pass noop provider to skip state root calculations for flashblocks (paradigmxyz#18161) * chore: Add 0x prefix and use macro (paradigmxyz#18156) * fix(optimism): find fb attrs in base fb (paradigmxyz#18164) * chore(deps): bump actions/upload-pages-artifact from 3 to 4 (paradigmxyz#18076) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * perf: optimize single-element collection creation (paradigmxyz#18168) * ci: Fix .PHONY declaration for install-reth-bench target in Makefile (paradigmxyz#18152) * fix: correct logical error in delete_outside_range error message (paradigmxyz#18031) * refactor: remove unnecessary PathBuf clone in CLI help generator (paradigmxyz#18172) * chore: simplify dev signed tx conversions (paradigmxyz#18171) * chore(deps): weekly `cargo update` (paradigmxyz#18174) Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> * perf(stages): optimize unwind operation by fetching headers instead full blocks (paradigmxyz#18139) * chore: avoid using hashmap hashers directly (paradigmxyz#18176) * feat: enhance engine tree metrics (paradigmxyz#18000) * perf(txpool): eliminate allocations in basefee enforcement (paradigmxyz#18162) * perf(optimism): use cached db in `FlashblockService` (paradigmxyz#18125) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * refactor(optimism): Extract responsibility to connect to a flashblock websocket stream (paradigmxyz#18158) * test ci on main * feat(examples): added txpoolExt_clearTxpool to existing example (paradigmxyz#18175) * fix: Pass prefix set from init_from_state_dump into compute_state_root (paradigmxyz#18185) * test(optimism): Cover successful decoding of websocket messages in `WsFlashBlockStream` (paradigmxyz#18163) * test(optimism): Cover the failure case of decoding a non-binary message in `WsFlashBlockStream` (paradigmxyz#18166) * test(optimism): Cover the case of stream returning errors in `WsFlashBlockStream` (paradigmxyz#18167) * test(optimism): Cover the case of repeatedly failing to connect to websocket in `WsFlashBlockStream` (paradigmxyz#18169) * dprint * fixes * fixes * fix: struct serialization to match actual fields (paradigmxyz#18189) * test(optimism): Test that streaming flashblocks from remote source is successful (paradigmxyz#18170) * fixes * fixes * fixes * resort * fixes * chore: simplify flashblocks poll logic (paradigmxyz#18194) Co-authored-by: julio4 <30329843+julio4@users.noreply.github.com> Co-authored-by: Roman Hodulák <roman.hodulak@polyglot-software.com> * fixes * chore: impl ExecutorTx for withtxenv (paradigmxyz#18202) * perf: prepare flashblock txs (paradigmxyz#18201) Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com> * fixes * local tar * try fork * try fork * fix: spawn flashblocks service as blocking (paradigmxyz#18214) * fixes * revert: "perf(txpool): eliminate allocations in basefee enforcement" (paradigmxyz#18215) * fix: incorrect blob fee comparison (paradigmxyz#18216) * fixes * add an extra check * add an extra check * fix(optimism): Prevent repeated executions of current flashblock sequence (paradigmxyz#18224) * fixes * try fix * added another check * added another check * test(optimism): Test that sequence stops before a gap (paradigmxyz#18228) * feat(optimism): Warn if `FlashBlockService` has stopped (paradigmxyz#18227) * chore: safe None check (paradigmxyz#18225) * chore: improve flashblock logs (paradigmxyz#18232) * fix(download): use updated merkle base URL (paradigmxyz#18236) * fixes * fixes * chore(engine): add better logs and spans for execution (paradigmxyz#18240) * chore(trie): use instrument instead of manual span (paradigmxyz#18239) * fix: filter zero storage values when computing withdrawals root in genesis header (paradigmxyz#18213) * enable by default * perf(reth-optimism-flashblocks): rm redundant clone (paradigmxyz#18196) * perf(txpool): eliminate allocations in basefee enforcement (paradigmxyz#18218) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fixes * docs: update urls in docs (paradigmxyz#18245) * feat(optimism): Respond to ping messages with pong in `WsFlashBlockStream` (paradigmxyz#18212) * removed one check is_some * with new release * test(optimism): Test that `WsFlashBlockStream` pongs a ping (paradigmxyz#18217) * fixes * fixes * chore: downgrade debug to trace for peer reputation logs (paradigmxyz#18250) * feat: added amsterdam to hardforks * use alloy config * chore: unify engine downloader targets (paradigmxyz#18248) * fix(optimism): Compare parent hash and latest hash to invalidate cached flashblock (paradigmxyz#18238) * feat(optimism): Decode text messages in `WsFlashBlockStream` (paradigmxyz#18257) * chore: remove redundant payload trait bounds (paradigmxyz#18262) * hivd * feat(optimism): Respond to close messages in `WsFlashBlockStream` (paradigmxyz#18256) * fix(optimism): Reconnect if ws stream ends in `WsFlashBlockStream` (paradigmxyz#18226) * test(optimism): Test that UTF-8 encoded messages are received in `WsFlashBlockStream` (paradigmxyz#18269) * chore: log prune settings on unwind (paradigmxyz#18270) * feat:db * new tarball * new tarball * chore: extract `validate_against_parent_gas_limit` into separate fn (paradigmxyz#18277) * perf: rm redundant collect (paradigmxyz#18281) * fixes * fixes * 100 * rerun * rerun * rerun * tracing * perf: build local pending block without updates (paradigmxyz#18271) * perf(db): do not heap-allocate the stage key per query (paradigmxyz#18284) * chore(txpool): add sanity tests for blob fee bit handling (paradigmxyz#18258) * fix: check prune checkpoints for unwind target limit (paradigmxyz#18263) * perf: optimize send raw batching (paradigmxyz#18280) * perf: specialize single batch request (paradigmxyz#18289) * chore: delist unused deps with `cargo-machete` (paradigmxyz#18259) * test(optimism): Test that close message is responded to in `WsFlashBlockStream` (paradigmxyz#18268) * tracing * fixes * perf: specialize validate_transactions_with_origin for task validator (paradigmxyz#18288) * feat(metrics): add `TxPoolValidatorMetrics` to track inflight validation jobs (paradigmxyz#18295) * fixes * fixes * feat(gpo): add default fee price argument (paradigmxyz#18297) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fixes * feat: introduce setting for delegated txs slots (paradigmxyz#18298) * feat: expose `EvmEnv` to `caller_gas_allowance` (paradigmxyz#18302) * perf: specialize len 1 (paradigmxyz#18307) * trace * chore: fix various typos in comments and documentation (paradigmxyz#18296) * perf(e2e-test-utils): optimize block checking by fetching header instead of full block (paradigmxyz#18254) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * feat: introduce maybe_pending method to StateProviderFactory (paradigmxyz#18260) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * chore: clippy happy (paradigmxyz#18310) * more trace * feat(download): support zst archives in reth download (paradigmxyz#18237) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * with new types * rerun * rerun * rerun * chore: introduce validationtask with capacity (paradigmxyz#18291) * chore(deps): weekly `cargo update` (paradigmxyz#18312) Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> * fix(rpc): error code `eth_sendRawTransactionSync` timeout (paradigmxyz#18252) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fixes * fixe * fixes * fixes * chore(trie): dont warn on blinded node reveals (paradigmxyz#18317) * refactor(optimism): Extract pending block building responsibility out of `FlashBlockService` (paradigmxyz#18247) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * rerun * refactor: change PendingPool and PendingTransaction visibility to pub (paradigmxyz#18267) * refactor(engine): persistence logic (paradigmxyz#18318) * feat(optimism): flashblock completed sequences (paradigmxyz#18272) * fix * feat(trie): Add helper sub-command (paradigmxyz#18301) * ci: pin Rust to 1.88 when building for Windows in Cross (paradigmxyz#18320) * docs(reth-bench): fix markdown (paradigmxyz#18322) * fix(bench): fix deadlock in test data generation (paradigmxyz#18321) * fix(stateless): verify_execution_witness doc for pre-state mismatch (paradigmxyz#18319) * fix: DB benches (paradigmxyz#18314) * chore: bump version 1.7.0 (paradigmxyz#18323) * chore: bump docs version 1.7.0 (paradigmxyz#18326) * docs: update public dashboards (paradigmxyz#18331) * chore(trie): use read-only db handle during repair-trie dry-runs (paradigmxyz#18328) * chore(deps): bump actions/setup-go from 5 to 6 (paradigmxyz#18332) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/github-script from 7 to 8 (paradigmxyz#18334) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/stale from 9 to 10 (paradigmxyz#18335) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: cache latest built payload (paradigmxyz#18324) * perf(merkle-stage): only fetch checkpoint in the branch that needs it (paradigmxyz#18339) * fix: properly compute genesis hash (paradigmxyz#18300) * feat(op-reth/flashblocks): subscribe to the flashblock sequences produced (paradigmxyz#18276) Co-authored-by: julio4 <30329843+julio4@users.noreply.github.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * perf(reth-engine-local): use VecDeque reduce removal operations (paradigmxyz#18198) * fix(ci): pin teku image to fix kurtosis-op build (paradigmxyz#18345) * fix(stages): implement entities checkpoint update in merkle stage unwind (paradigmxyz#18131) * rerurn * perf: use debug_assert for parked pool lookup (paradigmxyz#17712) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * refactor(revm): (paradigmxyz#18150) use hardfork activation helpers (paradigmxyz#18349) Co-authored-by: Waiting-Chai <1753609696@qq.com> * feat(stateless): Run EEST tests in stateless block validator & bug fixes (paradigmxyz#18140) Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * feat: support customizable RPC namespace parsers (paradigmxyz#18160) Co-authored-by: Federico Gimenez <federico.gimenez@gmail.com> * fix(prune): TransactionLookup pruning issues with pre-merge expiry (paradigmxyz#18348) * feat(op-sdk): custom precompiles (paradigmxyz#18350) * fix: fix search in vocs doc (paradigmxyz#18354) * merge * fix * fix: add is_osaka check before erroring in default_ethereum_payload (paradigmxyz#18355) * perf: optimize canonical_hashes_range with Vec::with_capacity pre-allocation + benchmark (paradigmxyz#18072) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * feat: add some ethapi builder fns (paradigmxyz#18358) * fix(docs): include .vocs to retain search-index (paradigmxyz#18363) * feat(engine): check header validity after invalid transaction (paradigmxyz#18356) * fix(engine): avoid block fetching inconsistencies for checks during reorgs (paradigmxyz#18368) * feat: bump hive eest tests (paradigmxyz#18013) * fix: check payload id (paradigmxyz#18370) * perf(trie): Use ParallelSparseTrie (if enabled) for storage tries (paradigmxyz#17959) * fixes * refactor!: more type-safety in cli (paradigmxyz#18375) * feat: add helper aliases for node adapters (paradigmxyz#18366) * fix: relax nonce gap rule if configured (paradigmxyz#18385) * fix(docs): disable jekyll which removes the search-index (paradigmxyz#18388) * fix(docs): mv search-index to dist from .vocs (paradigmxyz#18390) * fix: map EIP-7623 gas floor errors to expected exception type for test compatibility (paradigmxyz#18389) * feat: replace PendingBlockAndReceipts tuple with dedicated struct (paradigmxyz#18395) * fix: still use real chain id for no-op network (paradigmxyz#18382) * chore: use decode_2718_exact for recover raw txs (paradigmxyz#18381) * chore: fixed broken link in history-expiry.mdx (paradigmxyz#18400) * fix(e2e): persist accepted header in CheckPayloadAccepted and align timestamp (paradigmxyz#18275) Co-authored-by: Federico Gimenez <federico.gimenez@gmail.com> Co-authored-by: Federico Gimenez <fgimenez@users.noreply.github.com> * chore: update e2e-test-utils code owners (paradigmxyz#18397) * perf(db): reuse MDBX DBIs for the same tx (paradigmxyz#18292) * feat(txpool): break down queued transaction states into specific reasons (paradigmxyz#18106) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> * fix: dont update canon chain to ancestor for opstack (paradigmxyz#18410) * feat(observability): add phase-level observablity to newPayload processing (paradigmxyz#18308) Co-authored-by: YK <chiayongkang@hotmail.com> Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * perf: downsize mempool tx priority from `U256` to `u128` (paradigmxyz#18413) * chore(storage): remove unused `primed_dbis` (paradigmxyz#18415) * feat: fn recovered_tx to indexedTx (paradigmxyz#18421) * feat: add helper to PendingBlockAndReceipts (paradigmxyz#18423) * refactor and cleanup * fmt * fixes * clippy * clippy --------- Signed-off-by: 7suyash7 <suyashnyn1@gmail.com> Signed-off-by: tmel <tmel0103@gmail.com> Signed-off-by: tmelhao <tmel0103@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: 0xKitsune <77890308+0xKitsune@users.noreply.github.com> Co-authored-by: Roman Hodulák <roman.hodulak@polyglot-software.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> Co-authored-by: Hai | RISE <150876604+hai-rise@users.noreply.github.com> Co-authored-by: Dharm Singh <153282211+dharmvr1@users.noreply.github.com> Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com> Co-authored-by: nk_ysg <nk_ysg@163.com> Co-authored-by: leniram159 <leniram159@gmail.com> Co-authored-by: Andrea Simeoni <andreasimeoni84@gmail.com> Co-authored-by: Matus Kysel <MatusKysel@users.noreply.github.com> Co-authored-by: Suyash Nayan <89125422+7suyash7@users.noreply.github.com> Co-authored-by: Haotian <303518297@qq.com> Co-authored-by: tmel <tmel0103@gmail.com> Co-authored-by: Louis Brown <48462338+louisbrown0212@users.noreply.github.com> Co-authored-by: Max Bytefield <schoentgenaa@gmail.com> Co-authored-by: Eric Woolsey <ewoolsey@ualberta.ca> Co-authored-by: YK <chiayongkang@hotmail.com> Co-authored-by: Brian Picciano <me@mediocregopher.com> Co-authored-by: Jonas Bostoen <jonasbostoen@fastmail.com> Co-authored-by: quantix9 <quantix9@proton.me> Co-authored-by: Julio <30329843+julio4@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pepes <155114519+dennsikl@users.noreply.github.com> Co-authored-by: James Niken <155266991+dizer-ti@users.noreply.github.com> Co-authored-by: VolodymyrBg <aqdrgg19@gmail.com> Co-authored-by: David Klank <155117116+davidjsonn@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Co-authored-by: smileclown2024 <167074920+smileclown2024@users.noreply.github.com> Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com> Co-authored-by: Fynn <zcheng1004@gmail.com> Co-authored-by: Soubhik-10 <soubhiksmp2004@gmail.com> Co-authored-by: TMOT <166535397+Timosdev99@users.noreply.github.com> Co-authored-by: Brawn <nftdropped@gmail.com> Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com> Co-authored-by: Ivan Wang <wff19940326@gmail.com> Co-authored-by: Mablr <59505383+mablr@users.noreply.github.com> Co-authored-by: zhygis <5236121+Zygimantass@users.noreply.github.com> Co-authored-by: Femi Bankole <llordbanks500@gmail.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: kien-rise <157339831+kien-rise@users.noreply.github.com> Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com> Co-authored-by: radik878 <radikpadik76@gmail.com> Co-authored-by: theo <80177219+theochap@users.noreply.github.com> Co-authored-by: Federico Gimenez <fgimenez@users.noreply.github.com> Co-authored-by: かとり <SarahADavisA@gmail.com> Co-authored-by: malik <aremumalik05@gmail.com> Co-authored-by: Waiting-Chai <1753609696@qq.com> Co-authored-by: Ignacio Hagopian <jsign.uy@gmail.com> Co-authored-by: Federico Gimenez <federico.gimenez@gmail.com> Co-authored-by: Léa Narzis <78718413+lean-apple@users.noreply.github.com> Co-authored-by: Rez <rez@berachain.com> Co-authored-by: Yash Atreya <44857776+yash-atreya@users.noreply.github.com> Co-authored-by: Soubhik Singha Mahapatra <160333583+Soubhik-10@users.noreply.github.com> Co-authored-by: stevencartavia <112043913+stevencartavia@users.noreply.github.com> Co-authored-by: Cypher Pepe <125112044+cypherpepe@users.noreply.github.com> Co-authored-by: Snezhkko <snezhkodaria38@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

perf(trie): Use ParellelSparseTrie (if enabled) for storage tries

edac7d1

mediocregopher requested review from Rjected, fgimenez, mattsse, rkrasiuk and shekhirin as code owners August 20, 2025 12:12

mediocregopher added C-perf A change motivated by improving speed, memory usage or disk footprint A-trie Related to Merkle Patricia Trie implementation labels Aug 20, 2025

github-project-automation bot added this to Reth Tracker Aug 20, 2025

github-project-automation bot moved this to Backlog in Reth Tracker Aug 20, 2025

mediocregopher mentioned this pull request Aug 20, 2025

perf(trie): default ParallelSparseTrie to enabled (accounts only still) #17956

Merged

mediocregopher changed the title ~~perf(trie): Use ParellelSparseTrie (if enabled) for storage tries~~ perf(trie): Use ParallelSparseTrie (if enabled) for storage tries Aug 20, 2025

Rjected approved these changes Aug 20, 2025

View reviewed changes

github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Aug 20, 2025

Merge branch 'main' into mediocregopher/pst-storage-tries

30c220e

jenpaff linked an issue Aug 22, 2025 that may be closed by this pull request

Use parallel sparse trie for storage tries #17683

Closed

Fast-path in PST::root when there have been no changes

6445179

fynnss reviewed Aug 25, 2025

View reviewed changes

mediocregopher added 2 commits August 28, 2025 14:05

Add parallelism_threshold to ParallelSparseTrie

cf76441

Fast-path fix

7a1937d

mediocregopher requested a review from klkvr as a code owner August 28, 2025 12:29

mediocregopher added 2 commits August 28, 2025 14:29

Add --engine.parallel-sparse-trie-threshold

db6c0f2

Merge remote-tracking branch 'upstream/main' into mediocregopher/pst-…

501ef4c

…storage-tries

Rjected reviewed Aug 28, 2025

View reviewed changes

mediocregopher added 4 commits August 29, 2025 23:13

Merge branch 'main' into mediocregopher/pst-storage-tries

8f9401d

Revert "Add --engine.parallel-sparse-trie-threshold"

a006944

This reverts commit db6c0f2.

Split parallelism thresholds into a two-field struct, for added clarity

703b537

Merge remote-tracking branch 'upstream/main' into mediocregopher/pst-…

572653e

…storage-tries

PST default to no thresholds, set them in engine

51e154e

mediocregopher requested a review from Rjected September 1, 2025 11:05

Merge remote-tracking branch 'upstream/main' into mediocregopher/pst-…

6d40014

…storage-tries

mattsse approved these changes Sep 9, 2025

View reviewed changes

Rjected approved these changes Sep 10, 2025

View reviewed changes

Rjected added this pull request to the merge queue Sep 10, 2025

Merged via the queue into main with commit 967a6fb Sep 10, 2025
42 checks passed

Rjected deleted the mediocregopher/pst-storage-tries branch September 10, 2025 23:07

github-project-automation bot moved this from In Progress to Done in Reth Tracker Sep 10, 2025

DaniPopes mentioned this pull request Feb 4, 2026

feat(trie): remove SerialSparseTrie #21808

Merged

		/// A default cleared trie instance, which will be cloned when creating new tries.
		default_trie: SparseTrie<S>,

Conversation

mediocregopher commented Aug 20, 2025

Uh oh!

Rjected left a comment

Choose a reason for hiding this comment

Uh oh!

Rjected Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Rjected commented Aug 20, 2025

Uh oh!

mediocregopher commented Aug 22, 2025

Uh oh!

fynnss Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

mediocregopher Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

fynnss Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fynnss Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mediocregopher Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

fynnss Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

gakonst commented Aug 28, 2025

Uh oh!

mediocregopher commented Aug 28, 2025

Uh oh!

mediocregopher commented Aug 28, 2025

Uh oh!

Rjected left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fynnss commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mediocregopher commented Aug 29, 2025

Uh oh!

fynnss commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mediocregopher commented Sep 1, 2025

Uh oh!

mattsse left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fynnss Aug 26, 2025 •

edited

Loading

fynnss Aug 27, 2025 •

edited

Loading

Rjected left a comment •

edited

Loading

fynnss commented Aug 29, 2025 •

edited

Loading

fynnss commented Aug 29, 2025 •

edited

Loading