fix(engine): defer changeset static file truncation during reorgs by joshieDo · Pull Request #23291 · paradigmxyz/reth

joshieDo · 2026-03-30T19:31:05Z

Defers changeset static file truncation during RemoveBlocksAbove to the next SaveBlocks, preventing stale mmap panics when concurrent readers hold old memory-mapped handles.

Waits for all MDBX readers from the reorg era to drain (via oldest_reader_txnid) before applying the deferred truncation.

Follow-up: #23305 (pins RocksDB snapshot for cross-store read consistency, stacked on this PR)

During RemoveBlocksAbove, changeset static file prune strategies are now deferred to the next SaveBlocks commit. This prevents truncating memory-mapped files while concurrent readers (payload builders, RPC) may still hold stale handles, which caused a panic in NippyJar from reading garbage offsets on the truncated mmap. Headers, transactions, and receipts are still truncated immediately. Only changeset prunes are deferred since those are the segments read concurrently during state root computation. Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

github-actions · 2026-03-30T19:31:51Z

⚠️ Changelog not found.

A changelog entry is required before merging. We've generated a suggested changelog based on your changes:

Preview

---
reth-engine-tree: patch
reth-db-api: minor
reth-db: patch
reth-libmdbx: minor
reth-provider: minor
---

Added deferred changeset static file truncation during reorgs to prevent truncating memory-mapped files while concurrent readers (payload builders, RPC) may still hold stale handles. Added `oldest_reader_txnid` and `last_txnid` to the `Database` trait and implemented them for MDBX, and added `take_changeset_prunes`/`requeue_changeset_prunes` to `StaticFileWriter` to support deferring and re-applying changeset prunes after stale readers drain.

Add changelog to commit this to your branch.

joshieDo · 2026-03-30T19:33:05Z

+        debug_assert_eq!(
+            account, storage,
+            "account and storage changeset prunes must target the same block"
+        );
+        account.or(storage)


this should be the same block, unsure if a new type for both is necessary

Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

… changeset prune Adds oldest_reader_txnid() and last_txnid() to the Database trait, backed by MDBX's mi_latter_reader_txnid and mi_recent_txnid. Before applying a deferred changeset prune in SaveBlocks, the persistence thread now spins until all MDBX readers from the reorg era have completed. This ensures no reader holds a stale mmap handle when the changeset files are truncated. Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

Amp-Thread-ID: https://ampcode.com/threads/T-019d4435-3bfe-7739-b47f-fbcf6473a16a Co-authored-by: Amp <amp@ampcode.com>

This reverts commit 5036cc4.

Amp-Thread-ID: https://ampcode.com/threads/T-019d4435-3bfe-7739-b47f-fbcf6473a16a Co-authored-by: Amp <amp@ampcode.com>

Rjected

mostly nits, based on my understanding of the problem this fix should work. I think we should add some docs explaining why we would not get any new readers by the time we call save_blocks and wait for any final readers to complete

Rjected · 2026-03-31T17:51:21Z

    /// Last transaction ID
    #[inline]
    pub const fn last_txnid(&self) -> usize {
        self.0.mi_recent_txnid as usize
    }


can we document that this is the most recent txn id?

Rjected · 2026-03-31T17:55:21Z

+    ///
+    /// This only restores the queued prune-on-commit state. Safety still depends on the caller
+    /// having waited for stale MDBX readers to drain before the next commit executes the prune.
+    pub(crate) fn requeue_changeset_prunes(&self, last_block: BlockNumber) -> ProviderResult<()> {


can we rename this from requeue to maybe complete or something that mentions that this finalizes / flushes the queued prunes?

Rjected · 2026-03-31T18:05:11Z

+    /// commit.
+    ///
+    /// This is used by the deferred reorg-prune flow in the persistence service. Callers must
+    /// only reapply the returned prune after stale MDBX readers from the pre-unwind era have


Suggested change

/// only reapply the returned prune after stale MDBX readers from the pre-unwind era have

/// only apply the returned prune after stale MDBX readers from the pre-unwind era have

mattsse · 2026-04-01T08:35:51Z

+/// concurrent readers may still hold handles. The prune is applied at the start of the next
+/// `SaveBlocks`, after waiting for all MDBX readers from the reorg era to drain.
+#[derive(Debug)]
+struct DeferredChangesetPrune {


lets please not add private types to top of file :)

mattsse · 2026-04-01T08:38:31Z

+                    .oldest_reader_txnid()
+                    .is_some_and(|oldest| oldest < prune_txn)


is this perhaps too relaxed?

this would also include rpc related transactions?
and there's a risk that an ethgetproof or adjacent tx will be open for a while?
should we either introduce a counter, that after like 3 blocks or so we always do this?

github-project-automation Bot added this to Reth Tracker Mar 30, 2026

github-project-automation Bot moved this to Backlog in Reth Tracker Mar 30, 2026

joshieDo commented Mar 30, 2026

View reviewed changes

joshieDo and others added 2 commits March 30, 2026 20:38

fix: clippy warnings

f0bdf57

Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

fix: gate deferred changeset prune on storage_v2

6bc3dc0

Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

joshieDo marked this pull request as ready for review March 30, 2026 19:42

joshieDo requested review from Rjected, mattsse, mediocregopher, shekhirin and yongkangc as code owners March 30, 2026 19:42

joshieDo added C-bug An unexpected or incorrect behavior A-db Related to the database A-engine Related to the engine implementation labels Mar 30, 2026

joshieDo and others added 2 commits March 30, 2026 20:51

fix: collapse nested if for clippy

ce84c87

Amp-Thread-ID: https://ampcode.com/threads/T-019d3ffb-22ed-70d4-ad5c-23eeed6f0ad7 Co-authored-by: Amp <amp@ampcode.com>

joshieDo requested a review from gakonst as a code owner March 31, 2026 12:08

joshieDo changed the title ~~fix(engine): defer changeset static file truncation during reorgs~~ fix(storage): pin RocksDB snapshot for cross-store read consistency Mar 31, 2026

joshieDo marked this pull request as draft March 31, 2026 12:09

joshieDo force-pushed the joshie/defer-changeset-prune branch 3 times, most recently from c6a5cab to d1ecfa7 Compare March 31, 2026 12:27

joshieDo mentioned this pull request Mar 31, 2026

fix(storage): pin RocksDB snapshot for cross-store read consistency #23305

Closed

joshieDo changed the title ~~fix(storage): pin RocksDB snapshot for cross-store read consistency~~ fix(engine): defer changeset static file truncation during reorgs Mar 31, 2026

joshieDo and others added 3 commits March 31, 2026 15:19

fix(storage): reuse pinned RocksDB snapshot path

5036cc4

Amp-Thread-ID: https://ampcode.com/threads/T-019d4435-3bfe-7739-b47f-fbcf6473a16a Co-authored-by: Amp <amp@ampcode.com>

Revert "fix(storage): reuse pinned RocksDB snapshot path"

3f86ae3

This reverts commit 5036cc4.

docs(engine): clarify deferred changeset prune contracts

fc12649

Amp-Thread-ID: https://ampcode.com/threads/T-019d4435-3bfe-7739-b47f-fbcf6473a16a Co-authored-by: Amp <amp@ampcode.com>

joshieDo marked this pull request as ready for review March 31, 2026 17:09

Rjected reviewed Mar 31, 2026

View reviewed changes

mattsse reviewed Apr 1, 2026

View reviewed changes

joshieDo mentioned this pull request Apr 1, 2026

fix(provider): cap storage_v2 unwind history by MDBX tip #23335

Merged

joshieDo closed this Apr 2, 2026

github-project-automation Bot moved this from Backlog to Done in Reth Tracker Apr 2, 2026

emmajam deleted the joshie/defer-changeset-prune branch May 1, 2026 10:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(engine): defer changeset static file truncation during reorgs#23291

fix(engine): defer changeset static file truncation during reorgs#23291
joshieDo wants to merge 8 commits into
mainfrom
joshie/defer-changeset-prune

joshieDo commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

joshieDo Mar 30, 2026 •

edited

Loading

Uh oh!

Rjected left a comment

Uh oh!

Rjected Mar 31, 2026

Uh oh!

Rjected Mar 31, 2026

Uh oh!

Rjected Mar 31, 2026

Uh oh!

mattsse Apr 1, 2026

Uh oh!

mattsse Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	/// only reapply the returned prune after stale MDBX readers from the pre-unwind era have
	/// only apply the returned prune after stale MDBX readers from the pre-unwind era have

		.oldest_reader_txnid()
		.is_some_and(\|oldest\| oldest < prune_txn)

Conversation

joshieDo commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Changelog not found.

Uh oh!

joshieDo Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rjected left a comment

Choose a reason for hiding this comment

Uh oh!

Rjected Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Rjected Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Rjected Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

mattsse Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

mattsse Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joshieDo commented Mar 30, 2026 •

edited

Loading

github-actions Bot commented Mar 30, 2026 •

edited

Loading

joshieDo Mar 30, 2026 •

edited

Loading