feat(engine): include backpressure in reported persistence_wait, make wait-time mimic CL slot time#23308
Merged
mediocregopher merged 8 commits intoApr 1, 2026
Conversation
…_wait persistence_wait_us now always reports the total time spent waiting on persistence: channel queuing time from backpressure plus any explicit wait-for-persistence wait. An enqueued_at timestamp is recorded when the RethNewPayload message is sent and used to measure backpressure delay when the message is dequeued. The field is no longer optional — it is always populated even when wait-for-persistence is not requested. Also removes the reth_newPayload and wait-for-persistence toggles from the bench GitHub workflow, always using reth_newPayload and relying on backpressure for persistence timing. Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
Contributor
✅ Changelog found on PR. |
Member
|
|
Member
|
✅ Benchmark complete! View job Benchmark Results
100 big blocks Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
Sleep only the remaining time after block processing to fill the slot, so blocks are submitted at a fixed interval regardless of validation time. If validation exceeds the interval, the next block is submitted immediately. Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
…mantics Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
Member
|
✅ Benchmark complete! View job Benchmark Results
100 blocks Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
Member
|
✅ Benchmark complete! View job Benchmark Results
100 big blocks Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
Add wait_time to the summary JSON dict so it's available to all renderers. Displayed in the PR comment (via generate_markdown), the GitHub Actions job summary (via blocksLabel), and the Slack notification. Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
Member
|
✅ Benchmark complete! View job Benchmark ResultsWait time:
500 blocks Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
The metadata line below the comparison table now shows warmup blocks and wait-time alongside the block count, matching the job summary format. Both values are also stored in summary.json for use by the job summary and Slack notification renderers. Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com>
Member
|
✅ Benchmark complete! View job Benchmark Results
10 big blocks, 10 warmup, wait time: 10ms Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
mediocregopher
changed the title
mediocregopher
requested review from
DaniPopes,
Rjected,
gakonst,
mattsse,
shekhirin and
yongkangc
as code owners
…tence-wait-backpressure Amp-Thread-ID: https://ampcode.com/threads/T-019d4438-f509-7388-b646-59af336c6ca1 Co-authored-by: Amp <amp@ampcode.com> # Conflicts: # .github/scripts/bench-reth-run.sh
Member
|
✅ Benchmark complete! View job Benchmark Results
20 big blocks, 10 warmup, wait time: 100ms Wait Time BreakdownPersistence Wait
Trie Cache Update Wait
Execution Cache Update Wait
ChartsLatency, Throughput & Diff
Wait Time Breakdown
Gas vs Latency
Grafana Dashboard |
pepyakin
approved these changes
Apr 1, 2026
| /// Time spent waiting on persistence in microseconds, including both time spent | ||
| /// queued due to persistence backpressure and, when requested, the explicit wait | ||
| /// for in-flight persistence to complete. | ||
| pub persistence_wait_us: u64, |
Member
There was a problem hiding this comment.
Nit: why not break it down/split into persistence / backpressure?
Member
Author
There was a problem hiding this comment.
seemed like a YAGNI situation, i think with backpressure we won't really need wait-for-persistence going forward, and this is less changes to the plumbing
github-merge-queue Bot
pushed a commit
to tempoxyz/tempo
that referenced
this pull request
Apr 3, 2026
Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`f8efc76...c82d435`](paradigmxyz/reth@f8efc76...c82d435) 🔗 Amp thread: https://ampcode.com/threads/T-019d517e-981b-75fd-b3cf-8ad2455f38f6 **RPC** - Remove `0x` prefix from `admin_peers` id and return keccak256 node ID in `admin_nodeInfo.id` to match go-ethereum format ([#23318](paradigmxyz/reth#23318), [#23319](paradigmxyz/reth#23319)) - Apply count filter only after `after` is consumed in pagination ([#23338](paradigmxyz/reth#23338)) - Pre-allocate vectors in `eth_feeHistory` ([#23334](paradigmxyz/reth#23334)) - Add `bal` RPC methods ([#23330](paradigmxyz/reth#23330)) **Engine** - Include backpressure in reported `persistence_wait`, make wait-time mimic CL slot time ([#23308](paradigmxyz/reth#23308)) **Perf** - Reduce cacheline ping-pong in workers availability ([#23321](paradigmxyz/reth#23321)) **Net** - Avoid itertools `Format` panic in tracing log ([#23331](paradigmxyz/reth#23331)) **DB** - Cap `storage_v2` unwind history by MDBX tip ([#23335](paradigmxyz/reth#23335)) **Bench** - Add warmup step for big blocks mode ([#23323](paradigmxyz/reth#23323)) - Make payload handling ethereum-only ([#23324](paradigmxyz/reth#23324)) - Use repository instead of GitHub Actions cache to track hourly runs ([#23306](paradigmxyz/reth#23306)) - Upload nightly regression results to ClickHouse ([#23344](paradigmxyz/reth#23344)) **Chore** - Migrate `allow(clippy::` to `expect(clippy::` ([#23340](paradigmxyz/reth#23340)) - Remove unused return value from `dispatch_with_chunking` ([#23341](paradigmxyz/reth#23341)) - Update nixpkgs to 25.11 and refresh flake inputs ([#23342](paradigmxyz/reth#23342)) ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-019d517e-baf0-727b-a57e-0488240af973 - **Reth dependency bump**: All `reth-*` git dependencies updated from rev `f8efc76` to `c82d435` - **New `GetBlockAccessList` trait impl**: `TempoEthApi` now implements `GetBlockAccessList` (imported from `reth_rpc_eth_api::helpers::bal`), likely a new required trait added upstream in the reth update [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/23933027282) --------- Co-authored-by: Arsenii Kulikov <klkvrr@gmail.com>
This was referenced May 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
persistence_wait_us now always reports the total time spent waiting on persistence: channel queuing time from backpressure plus any explicit wait-for-persistence wait.
persistence_wait_us is no longer optional — it is always populated even when wait-for-persistence is not requested.
reth_newPayload and wait-for-persistence toggles are removed from the bench GitHub workflow, always using reth_newPayload and relying on backpressure for persistence timing.
wait-time in reth-bench now controls the interval between nP submissions excluding validation time, mimicking a CL slot time. If nP does not complete within the wait-time then the next nP is submitted immediately.