feat(engine): backpressure by pepyakin · Pull Request #23244 · paradigmxyz/reth

pepyakin · 2026-03-26T15:58:23Z

No description provided.

pepyakin · 2026-03-26T16:00:02Z

derek bench blocks=big otlp=false

decofe · 2026-03-26T16:00:15Z

cc @pepyakin

❌ Benchmark failed while running baseline benchmark (1/2). View logs

github-actions · 2026-03-26T16:00:48Z

⚠️ Changelog not found.

A changelog entry is required before merging. We've generated a suggested changelog based on your changes:

Preview

---
reth-engine-primitives: minor
reth-engine-tree: minor
reth-bench: minor
---

Added persistence backpressure to the engine tree: when the canonical-minus-persisted block gap reaches a configurable threshold (`DEFAULT_PERSISTENCE_BACKPRESSURE_THRESHOLD = 16`), incoming beacon messages (newPayload, forkchoiceUpdated) are stashed and processing stalls until persistence catches up. Added `backpressure_wait` timing to `NewPayloadTimings` and bench output so the wait is visible in metrics and benchmark CSV results.

Add changelog to commit this to your branch.

pepyakin · 2026-03-26T16:42:24Z

derek bench

decofe · 2026-03-26T16:42:40Z

cc @pepyakin

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 5 commits behind main. Consider rebasing for accurate results.

Metric	`main`	`pep/backpressure`	Change
Mean	22.95ms	23.15ms	+0.85% ❌ (±0.51%)
StdDev	13.38ms	13.98ms
P50	20.32ms	20.52ms	+0.95% ⚪ (±1.24%)
P90	34.24ms	34.33ms	+0.25% ⚪ (±1.89%)
P99	81.04ms	81.47ms	+0.53% ⚪ (±7.32%)
Mgas/s	1382.06	1378.09	-0.29% ⚪ (±0.41%)
Wall Clock	23.64s	23.84s	+0.85% ❌ (±0.53%)

500 blocks

Wait Time Breakdown

Persistence Wait

Metric	`main`	`pep/backpressure`
Mean	65.59ms	66.31ms
P50	0.00ms	0.00ms
P95	257.63ms	258.38ms

Trie Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.13ms	0.11ms
P50	0.00ms	0.00ms
P95	0.60ms	0.58ms

Execution Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.00ms	0.00ms
P50	0.00ms	0.00ms
P95	0.00ms	0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Wait Time Breakdown

Gas vs Latency

Grafana Dashboard

View real-time metrics

pepyakin · 2026-03-26T17:07:34Z

derek bench

decofe · 2026-03-26T17:07:48Z

cc @pepyakin

✅ Benchmark complete! View job

Benchmark Results

Metric	`main`	`pep/backpressure`	Change
Mean	22.90ms	22.89ms	-0.05% ⚪ (±0.35%)
StdDev	13.39ms	13.34ms
P50	20.46ms	20.37ms	-0.47% ⚪ (±1.23%)
P90	34.71ms	34.56ms	-0.45% ⚪ (±2.24%)
P99	77.44ms	76.86ms	-0.75% ⚪ (±3.85%)
Mgas/s	1385.81	1385.14	-0.05% ⚪ (±0.42%)
Wall Clock	23.61s	23.59s	-0.09% ⚪ (±0.39%)

500 blocks

Wait Time Breakdown

Persistence Wait

Metric	`main`	`pep/backpressure`
Mean	64.27ms	64.11ms
P50	0.00ms	0.00ms
P95	252.24ms	250.79ms

Trie Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.10ms	0.10ms
P50	0.00ms	0.00ms
P95	0.54ms	0.50ms

Execution Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.00ms	0.00ms
P50	0.00ms	0.00ms
P95	0.00ms	0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Wait Time Breakdown

Gas vs Latency

Grafana Dashboard

View real-time metrics

decofe · 2026-03-26T17:27:51Z

cc @pepyakin

✅ Benchmark complete! View job

Benchmark Results

Metric	`main`	`pep/backpressure`	Change
Mean	473.50ms	470.21ms	-0.70% ✅ (±0.37%)
StdDev	200.59ms	199.32ms
P50	500.59ms	498.76ms	-0.37% ⚪ (±1.86%)
P90	672.13ms	667.74ms	-0.65% ⚪ (±1.54%)
P99	949.53ms	945.80ms	-0.39% ⚪ (±1.13%)
Mgas/s	2030.91	2048.31	+0.86% ✅ (±0.49%)
Wall Clock	41.06s	40.77s	-0.71% ✅ (±0.37%)

43 big blocks

Wait Time Breakdown

Persistence Wait

Metric	`main`	`pep/backpressure`
Mean	517.54ms	517.96ms
P50	0.00ms	0.00ms
P95	1927.80ms	1929.22ms

Trie Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.00ms	0.00ms
P50	0.00ms	0.00ms
P95	0.00ms	0.00ms

Execution Cache Update Wait

Metric	`main`	`pep/backpressure`
Mean	0.00ms	0.00ms
P50	0.00ms	0.00ms
P95	0.00ms	0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Wait Time Breakdown

Gas vs Latency

Grafana Dashboard

View real-time metrics

mediocregopher · 2026-03-27T09:40:20Z

-        let safe_hash = B256::random();
-        let finalized_hash = B256::random();
-        let non_matching_hash = B256::random();
+        let head_hash = B256::with_last_byte(1);


What was this about?

mediocregopher · 2026-03-27T09:51:48Z

        }
    }

+    #[cfg(test)]


nit: let's just remove the cfg(test) and not have two methods

pepyakin · 2026-03-30T15:20:17Z

superseded by #23280

github-project-automation Bot added this to Reth Tracker Mar 26, 2026

github-project-automation Bot moved this to Backlog in Reth Tracker Mar 26, 2026

pepyakin added 5 commits March 26, 2026 17:03

engine: add persistence backpressure for beacon requests

b37a77f

engine: fix backpressure stall handling

8fbcf7f

engine-primitives: make forkchoice test deterministic

624a1eb

tidy up

32c55d3

engine: fix rebase CI fallout

5732c3c

pepyakin force-pushed the pep/backpressure branch from 931636d to 5732c3c Compare March 26, 2026 17:07

mediocregopher approved these changes Mar 27, 2026

View reviewed changes

github-project-automation Bot moved this from Backlog to In Progress in Reth Tracker Mar 27, 2026

pepyakin mentioned this pull request Mar 30, 2026

feat(engine): backpressure, take 2. #23280

Merged

pepyakin closed this Mar 30, 2026

github-project-automation Bot moved this from In Progress to Done in Reth Tracker Mar 30, 2026

emmajam deleted the pep/backpressure branch May 1, 2026 10:10

Conversation

pepyakin commented Mar 26, 2026

Uh oh!

pepyakin commented Mar 26, 2026

Uh oh!

decofe commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Changelog not found.

Uh oh!

pepyakin commented Mar 26, 2026

Uh oh!

decofe commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results

Persistence Wait

Trie Cache Update Wait

Execution Cache Update Wait

Charts

Grafana Dashboard

Uh oh!

pepyakin commented Mar 26, 2026

Uh oh!

decofe commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results

Persistence Wait

Trie Cache Update Wait

Execution Cache Update Wait

Charts

Grafana Dashboard

Uh oh!

decofe commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results

Persistence Wait

Trie Cache Update Wait

Execution Cache Update Wait

Charts

Grafana Dashboard

Uh oh!

mediocregopher Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

mediocregopher Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

pepyakin commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

decofe commented Mar 26, 2026 •

edited

Loading

github-actions Bot commented Mar 26, 2026 •

edited

Loading

decofe commented Mar 26, 2026 •

edited

Loading

decofe commented Mar 26, 2026 •

edited

Loading

decofe commented Mar 26, 2026 •

edited

Loading