Skip to content

perf(re-execute): relax executor reset thresholds#23617

Merged
emmajam merged 1 commit into
mainfrom
auto-opt/20260417-relax-executor-resets
Apr 20, 2026
Merged

perf(re-execute): relax executor reset thresholds#23617
emmajam merged 1 commit into
mainfrom
auto-opt/20260417-relax-executor-resets

Conversation

@emmajam
Copy link
Copy Markdown
Member

@emmajam emmajam commented Apr 17, 2026

Increase state cache size limit from 1M to 5M entries and executor lifetime from 120s to 600s. Preserves hot state caches longer during re-execute, reducing redundant DB lookups for frequently accessed accounts and storage slots.

Benchmark (dev-emma, 3 runs, blocks 21M–21.001M)

Baseline (main) Feature
Run 1 0.478 Ggas/s 0.520 Ggas/s
Run 2 0.486 Ggas/s 0.520 Ggas/s
Run 3 0.480 Ggas/s 0.512 Ggas/s
Mean 0.481 Ggas/s 0.517 Ggas/s

+7.5% ±1.5% (sig=good) — statistically significant at 95% CI via Welch's t-test.

Changes

  • crates/cli/commands/src/re_execute.rs: size_hint threshold 1M → 5M, executor_lifetime 120s → 600s

Increase state cache size limit from 1M to 5M entries and executor
lifetime from 120s to 600s. Preserves hot state caches longer during
re-execute, reducing redundant DB lookups for frequently accessed
accounts and storage slots.

Amp-Thread-ID: https://ampcode.com/threads/T-019d9b2e-e5ea-75ef-ac3d-106a3a28c17a
Co-authored-by: Amp <amp@ampcode.com>
@emmajam
Copy link
Copy Markdown
Member Author

emmajam commented Apr 17, 2026

50K block benchmark (dev-emma, 3 runs, blocks 21M–21.05M)

Baseline (main) Feature
Run 1 2.39 Ggas/s 2.73 Ggas/s
Run 2 2.49 Ggas/s 2.80 Ggas/s
Run 3 2.50 Ggas/s 2.82 Ggas/s
Mean 2.46 Ggas/s 2.78 Ggas/s

+13.1% ±3.5% (sig=good) — stronger effect at scale since more executor resets are avoided over 50K blocks.

@emmajam emmajam marked this pull request as ready for review April 17, 2026 16:08
@emmajam emmajam requested review from Rjected and mattsse as code owners April 17, 2026 16:08
Copy link
Copy Markdown
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems fine

let next_block = Arc::clone(&next_block);
tasks.spawn_blocking(move || {
let executor_lifetime = Duration::from_secs(120);
let executor_lifetime = Duration::from_secs(600);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this exceeds the 5min threshold we usually have but in this case it doesnt really matter and I believe we'll always encounter the size hint first

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just benching the change on tempo also

@github-project-automation github-project-automation Bot moved this from Backlog to In Progress in Reth Tracker Apr 18, 2026
@emmajam
Copy link
Copy Markdown
Member Author

emmajam commented Apr 20, 2026

Tempo benchmark — no improvement

Tested the same change on Tempo chain data (21 GB, blocks 14M–15M, dev-emma).

1K blocks (15M–15.001M):

Baseline Feature
Mean 2.26 Ggas/s 2.31 Ggas/s

1M blocks (14M–15M):

Baseline Feature
Run 1 86.62 Ggas/s 87.80 Ggas/s
Run 2 88.28 Ggas/s 87.54 Ggas/s
Run 3 88.56 Ggas/s 87.61 Ggas/s
Mean 87.82 Ggas/s 87.65 Ggas/s

No meaningful difference on Tempo. The chain is much smaller (21 GB vs 2.5 TB for Ethereum mainnet) so state fits in cache easily and executor resets are essentially free — the bottleneck is elsewhere.

@emmajam emmajam added this pull request to the merge queue Apr 20, 2026
Merged via the queue into main with commit 41fe41f Apr 20, 2026
35 checks passed
@emmajam emmajam deleted the auto-opt/20260417-relax-executor-resets branch April 20, 2026 09:48
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Reth Tracker Apr 20, 2026
crazywriter1 pushed a commit to crazywriter1/tempo that referenced this pull request Apr 23, 2026
Automated nightly update of reth dependencies from `paradigmxyz/reth`
main branch.

## Upstream reth changes


[`98ebc34...7839f3d`](paradigmxyz/reth@98ebc34...7839f3d)

🔗 Amp thread:
https://ampcode.com/threads/T-019db884-a7c3-738b-8f38-cc04f8942d8a
**Engine**
- Suppress persistence during payload building
([#23618](paradigmxyz/reth#23618))
- Align Amsterdam endpoint validation
([#23625](paradigmxyz/reth#23625))
- Revert [#23541](paradigmxyz/reth#23541) and
[#23578](paradigmxyz/reth#23578)
([#23646](paradigmxyz/reth#23646))
- Let consensus impls control which errors are transient
([#23668](paradigmxyz/reth#23668))
- Configure invalid header cache hit eviction
([#23670](paradigmxyz/reth#23670))

**Perf**
- Relax executor reset thresholds for re-execute
([#23617](paradigmxyz/reth#23617))
- Replace `BTreeMap` with `imbl::OrdMap` in `BestTransactions`
([#23621](paradigmxyz/reth#23621))
- Avoid reopening `.csoff` on every changeset lookup
([#23687](paradigmxyz/reth#23687))
- Disable read tx timeout during re-execute
([#23680](paradigmxyz/reth#23680))

**P2P / Net**
- Add snap/2 wire helpers and messages
([#23611](paradigmxyz/reth#23611))
- Optionally fetch BAL with full blocks
([#23629](paradigmxyz/reth#23629))
- Discv5 enabled by default
([#23686](paradigmxyz/reth#23686))

**DB**
- Add `reth db migrate-v2` for v1→v2 storage migration
([#23422](paradigmxyz/reth#23422))
- Detect and warn about ZFS
([#23685](paradigmxyz/reth#23685))

**BAL**
- Scaffold BAL store abstraction
([#23596](paradigmxyz/reth#23596))
- Enable BAL building in ethereum payload
([#23597](paradigmxyz/reth#23597))
- Add parallelization and batch IO flags
([#23663](paradigmxyz/reth#23663))

**Refactor**
- Make `WorkerPool` lazy by default
([#23627](paradigmxyz/reth#23627))
- Encapsulate state fetching in db provider
([#23656](paradigmxyz/reth#23656))
- Remove `TrieNodeProvider`
([#23658](paradigmxyz/reth#23658))
- Unify opaque consensus error helpers
([#23669](paradigmxyz/reth#23669))

**Payload**
- Add gas limit and slot number to `BlockOrPayload`
([#23624](paradigmxyz/reth#23624),
[#23626](paradigmxyz/reth#23626))

**Bench**
- Add CLI flag to fetch balances by default; require local benchmark
data ([#23655](paradigmxyz/reth#23655),
[#23679](paradigmxyz/reth#23679))

**Deps**
- Bump alloy crates to 2.0.1
([#23677](paradigmxyz/reth#23677)),
rustls-webpki
([#23681](paradigmxyz/reth#23681)), weekly
`cargo update`
([#23628](paradigmxyz/reth#23628))

**Testing**
- Remove unsafe `env::set_var(RUST_LOG)` from tests
([#23672](paradigmxyz/reth#23672))
- Address nightly clippy warnings
([#23630](paradigmxyz/reth#23630))

## Migrations

🔗 Amp thread:
https://ampcode.com/threads/T-019db884-dc46-71f5-a823-00c3a16191d4
- **Reth dependency bump**: All `reth-*` git dependencies updated from
rev `98ebc34` to `7839f3d`
- **Alloy version bump**: `alloy-*` crates updated from `2.0.0` to
`2.0.1`; `alloy-evm` changed from `0.33.2` to `0.33.0`
- **`ConsensusError::Other` → `ConsensusError::msg`**: All
`ConsensusError::Other(...)` calls migrated to
`ConsensusError::msg(...)`, which accepts `&str`/`impl Display` directly
instead of requiring `String` (removes `.to_string()` calls for string
literals)
- **`deny.toml` license exceptions**: Added MPL-2.0 exceptions for
`bitmaps`, `imbl`, and `imbl-sized-chunks` (new transitive dependencies)

[GitHub
Workflow](https://github.com/tempoxyz/tempo/actions/runs/24816009191)

---------

Co-authored-by: Alexey Shekhirin <github@shekhirin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants