perf(engine): include backpressure in newPayload latency metric#23541
Merged
mediocregopher merged 2 commits intoApr 15, 2026
Conversation
The reth_newPayload latency timing previously only measured on_new_payload execution, starting the clock after backpressure and persistence waits had already completed. This hid real-world engine responsiveness issues from benchmark results. Now latency = enqueued_at.elapsed() - explicit_persistence_wait, so backpressure is included in all headline metrics (Mean, P50, P90, P99, Wall Clock) while explicit persistence wait remains excluded. Also adds persistence wait as a row in the main benchmark comparison table with significance testing, so regressions in persistence backpressure are visible at a glance. Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019d9155-a132-7554-8e20-9766d1f0752d Co-authored-by: Amp <amp@ampcode.com>
Member
Author
|
derek bench blocks=10 warmup=0 abba=false |
Member
|
❌ Benchmark failed while running baseline benchmark (1/2). View logs |
This was referenced Apr 15, 2026
Member
Author
|
derek bench blocks=10 warmup=1 abba=false |
Member
|
✅ Benchmark complete! View job Benchmark Results
10 blocks, 1 warmup Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsGrafana Dashboard |
Adds the Persist Wait metric to the shared metricRows() function in bench-utils.js, which feeds both the Slack notification table and the GitHub Actions job summary. Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019d9155-a132-7554-8e20-9766d1f0752d Co-authored-by: Amp <amp@ampcode.com>
Member
Author
|
derek bench blocks=10 warmup=1 abba=false |
Member
|
✅ Benchmark complete! View job Benchmark Results
10 blocks, 1 warmup Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsGrafana Dashboard |
pepyakin
approved these changes
Apr 15, 2026
This was referenced Apr 16, 2026
mediocregopher
added a commit
that referenced
this pull request
Apr 20, 2026
This reverts commit 93b2201 and commit cf29b3f. Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Amp-Thread-ID: https://ampcode.com/threads/T-019daa9e-42a5-74da-a301-2dcd4474bf17 Co-authored-by: Amp <amp@ampcode.com>
This was referenced Apr 20, 2026
crazywriter1
pushed a commit
to crazywriter1/tempo
that referenced
this pull request
Apr 23, 2026
Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`98ebc34...7839f3d`](paradigmxyz/reth@98ebc34...7839f3d) 🔗 Amp thread: https://ampcode.com/threads/T-019db884-a7c3-738b-8f38-cc04f8942d8a **Engine** - Suppress persistence during payload building ([#23618](paradigmxyz/reth#23618)) - Align Amsterdam endpoint validation ([#23625](paradigmxyz/reth#23625)) - Revert [#23541](paradigmxyz/reth#23541) and [#23578](paradigmxyz/reth#23578) ([#23646](paradigmxyz/reth#23646)) - Let consensus impls control which errors are transient ([#23668](paradigmxyz/reth#23668)) - Configure invalid header cache hit eviction ([#23670](paradigmxyz/reth#23670)) **Perf** - Relax executor reset thresholds for re-execute ([#23617](paradigmxyz/reth#23617)) - Replace `BTreeMap` with `imbl::OrdMap` in `BestTransactions` ([#23621](paradigmxyz/reth#23621)) - Avoid reopening `.csoff` on every changeset lookup ([#23687](paradigmxyz/reth#23687)) - Disable read tx timeout during re-execute ([#23680](paradigmxyz/reth#23680)) **P2P / Net** - Add snap/2 wire helpers and messages ([#23611](paradigmxyz/reth#23611)) - Optionally fetch BAL with full blocks ([#23629](paradigmxyz/reth#23629)) - Discv5 enabled by default ([#23686](paradigmxyz/reth#23686)) **DB** - Add `reth db migrate-v2` for v1→v2 storage migration ([#23422](paradigmxyz/reth#23422)) - Detect and warn about ZFS ([#23685](paradigmxyz/reth#23685)) **BAL** - Scaffold BAL store abstraction ([#23596](paradigmxyz/reth#23596)) - Enable BAL building in ethereum payload ([#23597](paradigmxyz/reth#23597)) - Add parallelization and batch IO flags ([#23663](paradigmxyz/reth#23663)) **Refactor** - Make `WorkerPool` lazy by default ([#23627](paradigmxyz/reth#23627)) - Encapsulate state fetching in db provider ([#23656](paradigmxyz/reth#23656)) - Remove `TrieNodeProvider` ([#23658](paradigmxyz/reth#23658)) - Unify opaque consensus error helpers ([#23669](paradigmxyz/reth#23669)) **Payload** - Add gas limit and slot number to `BlockOrPayload` ([#23624](paradigmxyz/reth#23624), [#23626](paradigmxyz/reth#23626)) **Bench** - Add CLI flag to fetch balances by default; require local benchmark data ([#23655](paradigmxyz/reth#23655), [#23679](paradigmxyz/reth#23679)) **Deps** - Bump alloy crates to 2.0.1 ([#23677](paradigmxyz/reth#23677)), rustls-webpki ([#23681](paradigmxyz/reth#23681)), weekly `cargo update` ([#23628](paradigmxyz/reth#23628)) **Testing** - Remove unsafe `env::set_var(RUST_LOG)` from tests ([#23672](paradigmxyz/reth#23672)) - Address nightly clippy warnings ([#23630](paradigmxyz/reth#23630)) ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-019db884-dc46-71f5-a823-00c3a16191d4 - **Reth dependency bump**: All `reth-*` git dependencies updated from rev `98ebc34` to `7839f3d` - **Alloy version bump**: `alloy-*` crates updated from `2.0.0` to `2.0.1`; `alloy-evm` changed from `0.33.2` to `0.33.0` - **`ConsensusError::Other` → `ConsensusError::msg`**: All `ConsensusError::Other(...)` calls migrated to `ConsensusError::msg(...)`, which accepts `&str`/`impl Display` directly instead of requiring `String` (removes `.to_string()` calls for string literals) - **`deny.toml` license exceptions**: Added MPL-2.0 exceptions for `bitmaps`, `imbl`, and `imbl-sized-chunks` (new transitive dependencies) [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/24816009191) --------- Co-authored-by: Alexey Shekhirin <github@shekhirin.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.






The
reth_newPayloadlatency timing previously only measuredon_new_payloadexecution, starting the clock after backpressure and persistence waits had already completed. This hid real-world engine responsiveness issues from benchmark results — a change that caused 6x persistence backpressure could still show as a 7% improvement in all headline metrics.Now
latency = enqueued_at.elapsed() - explicit_persistence_wait, so backpressure (time the message spent queued because the engine was busy) is included in all headline metrics (Mean, P50, P90, P99, Wall Clock) while explicit persistence wait (the deliberate wait for previous block to persist) remains excluded.Also adds Persist Wait as a row in the main benchmark comparison table with significance testing, so regressions in persistence throughput are visible at a glance instead of buried in a collapsed section.
Prompted by: mediocregopher