Skip to content

feat(engine): support dual-frontier partial trie persistence#24100

Draft
mediocregopher wants to merge 99 commits into
mainfrom
mediocregopher/partial-persistence-clean
Draft

feat(engine): support dual-frontier partial trie persistence#24100
mediocregopher wants to merge 99 commits into
mainfrom
mediocregopher/partial-persistence-clean

Conversation

@mediocregopher
Copy link
Copy Markdown
Member

@mediocregopher mediocregopher commented May 10, 2026 •
edited
Loading

Tracks separate durable frontiers for block data and state/trie data so persistence can defer trie writes while keeping reorg/unwind and historical overlay paths correct.

mediocregopher and others added 30 commits April 16, 2026 12:38
@mediocregopher
feat(trie): add disjoint_by_keys for sorted overlays
9883b71 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9639-935f-73a4-ba50-7682a7b9aca0
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
feat(stages): add partial finish checkpoint field
037828f 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9639-935f-73a4-ba50-7682a7b9aca0
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
feat(stages): move partial trie progress into finish checkpoint
b97544a 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d968b-970d-735e-844c-b83ff245ce05
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(node): unwind startup to partial trie checkpoint
761acad 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d968b-970d-735e-844c-b83ff245ce05
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): simplify save_blocks trie masking API
e2bd518 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9728-846b-7418-8a69-ddbba65ce656
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
docs(provider): explain save_blocks trie masking range
84b5205 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9728-846b-7418-8a69-ddbba65ce656
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(engine): raise persistence defaults
ca35283 
Raise the default persistence threshold to 10 and derive the default backpressure threshold from the persistence and in-memory thresholds. Enable alloy getrandom for reth-engine-primitives tests so the existing B256::random() test coverage keeps compiling.

Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9a2b-716e-774d-8db7-b0308fa96a23
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(provider): persist partial trie finish checkpoint
9e38dde 
Make masked save_blocks persistence track partial trie progress in the Finish checkpoint and switch the trie-masking API to a masked-suffix start index so the final block is always covered by the mask when partial trie persistence is used.

Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9a2b-716e-774d-8db7-b0308fa96a23
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(engine): retain partial trie suffix after persistence
ca20cc1 
Return finish-stage partial trie progress from the persistence thread and keep blocks above that trie boundary resident in memory after persistence completes.

This preserves the old prune-through-tip behavior when the partial trie matches the persisted tip or is absent.

Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019d9ab4-1eab-713f-9d5d-71903b7bc724
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
feat(engine): add dual-frontier persistence planning
c757a31 
Co-Authored-By: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
@mediocregopher
refactor(chain-state): derive lazy overlay anchor from blocks
be4e8cd 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dab40-7d65-709b-90d6-98965c5c6a65
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(chain-state): address lazy overlay review feedback
cacb69a 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dab40-7d65-709b-90d6-98965c5c6a65
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(chain-state): cache lazy overlays by anchor
ebfaa6f 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dab6f-7cee-755e-9f9c-309ae0b8517c
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): resolve lazy overlay anchors at use time
5041d55 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dab6f-7cee-755e-9f9c-309ae0b8517c
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): separate overlay anchors from revert state
812e479 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019daba4-3598-758d-8771-cb5db784db81
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): infer overlay anchors from sources
5036eb5 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019daba4-3598-758d-8771-cb5db784db81
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): thread explicit requested anchors
b5ad001 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019daba4-3598-758d-8771-cb5db784db81
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(provider): anchor overlay state providers by hash
d92ad5a 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dafa5-05ee-739e-a0b6-037cc30e4a0e
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(provider): pass overlay anchors via constructor
134a7f3 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019daba4-3598-758d-8771-cb5db784db81
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(engine): anchor state-root test overlay factory
7db14d0 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dafbb-3571-73ef-ae67-43c242e2bf23
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(trie): initialize test overlay anchors
45db5e0 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dafbb-3571-73ef-ae67-43c242e2bf23
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(provider): satisfy overlay lint checks
ffb0587 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dafbb-3571-73ef-ae67-43c242e2bf23
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
Merge branch 'main' into mediocregopher/lazyoverlay-refactor
87b5240 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019db57a-7f08-7761-9a50-27a2a5c8f917
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(engine): update sparse trie overlay factory test
d5169ed 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019db57a-7f08-7761-9a50-27a2a5c8f917
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
style(engine): format sparse trie overlay test
c4d0949 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019db57a-7f08-7761-9a50-27a2a5c8f917
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
Merge branch 'main' into mediocregopher/lazyoverlay-refactor
6e8dbe3 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dbc30-68d0-76b8-a32c-e1173122ca48
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(trie): remove unused parallel test dependency
b60758e 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dbc30-68d0-76b8-a32c-e1173122ca48
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
fix(ci): clean bench checkouts and lock cargo builds
31d0c78 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dbee7-f576-769d-923c-dfdfe0a40858
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
refactor(provider): require overlay builder anchor hash
b6eec2e 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dbf08-3c9d-736b-9b47-3070f2bf2a54
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
merge(engine): bring lazy overlay refactor into partial persistence
843b5a8 
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019dcdca-2c60-711c-b1b8-7a6001a950fd
Co-authored-by: Amp <amp@ampcode.com>
@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 13, 2026

derek bench big-blocks blocks=50 baseline-args="--engine.persistence-threshold=13 --engine.persistence-backpressure-threshold=20" feature-args="--engine.persistence-threshold=13 --engine.num-state-masking-blocks=10 --engine.persistence-backpressure-threshold=20" slack=never samply

@decofe
Copy link
Copy Markdown
Member

decofe commented May 13, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 9 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 742.79ms 841.50ms +13.29% ❌ (±3.90%)
StdDev 112.34ms 187.42ms
P50 725.34ms 791.64ms +9.14% ❌ (±3.49%)
P90 920.19ms 1180.79ms +28.32% ❌ (±13.90%)
P99 1134.77ms 1485.09ms +30.87% ❌ (±14.91%)
Mgas/s 1402.28 1263.30 -9.91% ❌ (±2.27%)
Wall Clock 74.80s 84.94s +13.56% ❌ (±4.05%)
Persist Wait 105.17ms 10.14ms -90.35% ✅ (±82.63%)

50 big blocks, driver: txgen, 12 warmup

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 105.17ms 10.14ms
P50 0.06ms 0.06ms
P95 1139.60ms 0.18ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Samply Profiles

Grafana Dashboard

View real-time metrics

@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 17, 2026

derek bench big-blocks blocks=500 baseline-args="--engine.persistence-threshold=11 --engine.persistence-backpressure-threshold=24" feature-args="--engine.persistence-threshold=21 --engine.num-state-masking-blocks=10 --engine.persistence-backpressure-threshold=42"

@decofe
Copy link
Copy Markdown
Member

decofe commented May 17, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 28 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 679.33ms 777.94ms +14.52% ❌ (±1.76%)
StdDev 117.56ms 205.58ms
P50 653.17ms 718.39ms +9.98% ❌ (±1.44%)
P90 827.51ms 1079.88ms +30.50% ❌ (±7.23%)
P99 1098.46ms 1546.90ms +40.82% ❌ (±14.02%)
Mgas/s 1540.04 1380.50 -10.36% ❌ (±1.08%)
Wall Clock 704.59s 810.85s +15.08% ❌ (±2.46%)
Persist Wait 0.99ms 1.20ms +21.90% ⚪ (±147.93%)

500 big blocks, 125 warmup

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.99ms 1.20ms
P50 0.04ms 0.04ms
P95 0.08ms 0.08ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.08ms 0.10ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Grafana Dashboard

View real-time metrics

Brian added 2 commits May 18, 2026 17:58
Merge remote-tracking branch 'origin/main' into mediocregopher/partia…
f28d145 
…l-persistence-clean

# Conflicts:
#	crates/chain-state/src/lazy_overlay.rs
#	crates/engine/tree/src/tree/mod.rs
#	crates/storage/provider/src/providers/state/overlay.rs
fix(engine): restore partial persistence merge logic
1d285fe
@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 18, 2026

derek bench big-blocks baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 18, 2026 •
edited
Loading

cc @mediocregopher

❌ Benchmark failed while running benchmarks. View logs

@paradigmxyz paradigmxyz deleted a comment from decofe May 18, 2026
@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 18, 2026

derek bench big-blocks baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 18, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 1 commit behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 805.55ms 814.14ms +1.07% ❌ (±0.27%, floor 0.70%)
P50 740.10ms 762.89ms +3.08% ❌ (±0.93%, floor 0.70%)
P90 992.54ms 1019.24ms +2.69% ❌ (±1.12%, floor 1.35%)
P99 1821.53ms 1426.92ms -21.66% ✅ (±2.32%, floor 5.00%)
Mgas/s 1317.67 1287.71 -2.27% ❌ (±0.26%, floor 0.45%)
Wall Clock 537.26s 535.45s -0.34% ⚪ (±0.27%, floor 0.70%)
Persist Wait 0.27ms 16.07ms +5806.97% ❌ (±411.12%, floor 5.00%)

65 big blocks, 16 warmup, 10 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.27ms 16.07ms
P50 0.05ms 0.05ms
P95 0.09ms 0.10ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.45ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.01ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Grafana Dashboard

View real-time metrics

@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 19, 2026

derek bench big-blocks baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 19, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 8 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 802.57ms 765.13ms -4.66% ✅ (±0.35%, floor 0.70%)
P50 738.29ms 725.17ms -1.78% ✅ (±1.05%, floor 0.70%)
P90 998.00ms 971.07ms -2.70% ✅ (±0.68%, floor 1.35%)
P99 1907.35ms 1354.39ms -28.99% ✅ (±1.83%, floor 5.00%)
Mgas/s 1323.40 1364.76 +3.13% ✅ (±0.36%, floor 0.45%)
Wall Clock 535.24s 501.53s -6.30% ✅ (±0.34%, floor 0.70%)
Persist Wait 0.28ms 106.31ms +38327.50% ❌ (±761.41%, floor 5.00%)

65 big blocks, 16 warmup, 10 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.28ms 106.31ms
P50 0.05ms 0.04ms
P95 0.09ms 843.66ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.40ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Grafana Dashboard

View real-time metrics

decofe and others added 4 commits May 19, 2026 08:52
@decofe
fix engine backpressure default parsing
519750e
fix(engine): adjust backpressure default parsing
c9faab4
docs: update generated cli reference
d04eef4
Merge remote-tracking branch 'origin/pr/24334' into mediocregopher/pa…
2b3988e 
…rtial-persistence-clean

# Conflicts:
#	crates/node/core/src/args/engine.rs
#	crates/storage/provider/src/providers/database/provider.rs
#	docs/vocs/docs/pages/cli/reth/node.mdx
@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 19, 2026

derek bench big-blocks baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 19, 2026 •
edited
Loading

cc @mediocregopher

❌ Benchmark failed while building binaries. View logs

@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 19, 2026

derek bench big-blocks baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 19, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 2 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 809.46ms 800.99ms -1.05% ✅ (±0.17%, floor 0.70%)
P50 739.74ms 746.26ms +0.88% ⚪ (±1.01%, floor 0.70%)
P90 1007.25ms 1021.79ms +1.44% ⚪ (±1.49%, floor 1.35%)
P99 1854.35ms 1363.12ms -26.49% ✅ (±5.01%, floor 5.00%)
Mgas/s 1311.26 1309.79 -0.11% ⚪ (±0.21%, floor 0.45%)
Wall Clock 540.06s 526.49s -2.51% ✅ (±0.16%, floor 0.70%)
Persist Wait 0.06ms 0.11ms +83.75% ⚪ (±114.88%, floor 5.00%)

65 big blocks, 16 warmup, 10 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.06ms 0.11ms
P50 0.05ms 0.05ms
P95 0.08ms 0.09ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.67ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Grafana Dashboard

View real-time metrics

@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 19, 2026

derek bench big-blocks blocks=300 run-pairs=6 bal=true baseline-args="--engine.persistence-threshold=5" feature-args="--engine.persistence-threshold=11 --engine.num-state-masking-blocks=6" slack=never

@decofe
Copy link
Copy Markdown
Member

decofe commented May 19, 2026 •
edited
Loading

cc @mediocregopher

✅ Benchmark complete! View job

Benchmark Results

⚠️ Feature is 6 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/partial-persistence-clean Change
Mean 282.19ms 314.75ms +11.54% ❌ (±0.76%, floor 0.70%)
P50 269.25ms 290.56ms +7.92% ❌ (±0.97%, floor 0.70%)
P90 335.18ms 415.85ms +24.07% ❌ (±0.91%, floor 1.35%)
P99 546.17ms 576.99ms +5.64% ⚪ (±2.07%, floor 5.00%)
Mgas/s 3720.57 3382.07 -9.10% ❌ (±0.70%, floor 0.45%)
Wall Clock 523.45s 582.38s +11.26% ❌ (±0.73%, floor 0.70%)
Persist Wait 149.06ms 99.53ms -33.22% ✅ (±3.65%, floor 5.00%)

300 big blocks, 75 warmup, 6 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/partial-persistence-clean
Mean 149.06ms 99.53ms
P50 0.05ms 0.06ms
P95 1253.71ms 896.54ms

Trie Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.11ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/partial-persistence-clean
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Grafana Dashboard

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Grafana Dashboard

View real-time metrics

@debjit-bw
Copy link
Copy Markdown
Contributor

debjit-bw commented May 27, 2026

hi, is this still active?

@mediocregopher
Copy link
Copy Markdown
Member Author

mediocregopher commented May 27, 2026

hi @debjit-bw yes this is still active, there's some other optimizations which I'm working on ahead of this to make it viable, e.g. #24355

@debjit-bw
Copy link
Copy Markdown
Contributor

debjit-bw commented May 27, 2026

thanks @mediocregopher, then i will be pinning this commit in a PoC repo (where we use reth as an sdk), since it will eventually made into main

Merge branch 'main' of https://github.com/paradigmxyz/reth into medio…
6e4a0ae 
…cregopher/partial-persistence-clean

# Conflicts:
#	crates/node/builder/src/launch/common.rs
#	crates/node/core/src/args/engine.rs
#	docs/vocs/docs/pages/cli/reth/node.mdx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Rjected Rjected Awaiting requested review from Rjected Rjected will be requested when the pull request is marked ready for review Rjected is a code owner

@shekhirin shekhirin Awaiting requested review from shekhirin shekhirin will be requested when the pull request is marked ready for review shekhirin is a code owner

@joshieDo joshieDo Awaiting requested review from joshieDo joshieDo will be requested when the pull request is marked ready for review joshieDo is a code owner

@mattsse mattsse Awaiting requested review from mattsse mattsse will be requested when the pull request is marked ready for review mattsse is a code owner

@klkvr klkvr Awaiting requested review from klkvr klkvr will be requested when the pull request is marked ready for review klkvr is a code owner

@gakonst gakonst Awaiting requested review from gakonst gakonst will be requested when the pull request is marked ready for review gakonst is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@mediocregopher mediocregopher

Labels

A-db Related to the database A-engine Related to the engine implementation A-trie Related to Merkle Patricia Trie implementation

Projects

Reth Tracker
Status: Backlog

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mediocregopher @decofe @debjit-bw