feat(trie): MerkleChangeSets pipeline sync stage#18809
Merged
mediocregopher merged 28 commits into18460-trie-changesetsfrom Oct 3, 2025
Merged
feat(trie): MerkleChangeSets pipeline sync stage#18809mediocregopher merged 28 commits into18460-trie-changesetsfrom
mediocregopher merged 28 commits into18460-trie-changesetsfrom
Conversation
When implementing trie changesets we'll need to be able to query the HashedPostState revert just for specific blocks (in order to compute their PrefixSets). This allows for doing that with minimal other changes. A range helper for BlockNumberAddress is added for convenience.
These methods will be used in various places when it's necessary to delete changeset data. This includes pruning, unwinding, and also when populating the data during pipeline sync in certain cases. A new type BlockNumberHashedAddressRange is implemented for convenience.
This allows for passing optional overlays into the `write_trie_changesets` and `write_storage_trie_changesets` provider methods. The overlay is a TrieUpdates which is used to augment the state of the trie db tables. Using the overlay we can write changesets as if the DB is at a previous block, which will be used during pipeline sync. Implementing this change required refactoring the StorageTrieCurrentValuesIter utility to accept a TrieCursor rather than a normal DbCursor. It also required implementing a TrieCursorIter which wraps a TrieCursor into an Iterator, for passing in to `storage_trie_wiped_changeset_iter`. Using both of these changes we could use an InMemoryTrieCursor instead of a direct db cursor.
These methods will be used in various places when it's necessary to delete changeset data. This includes pruning, unwinding, and also when populating the data during pipeline sync in certain cases. A new type BlockNumberHashedAddressRange is implemented for convenience.
…diocregopher/clear-trie-changesets
…er/18464-trie-cs-pipeline
github-actions
bot
added
A-db
mediocregopher
changed the title
mattsse
reviewed
Oct 1, 2025
Collaborator
mattsse
left a comment
mattsse
left a comment
There was a problem hiding this comment.
not the expert when it comes to lower level merkle stuff, but I believe I could follow along here and thanks to the docs all of this makes sense to me
still need @Rjected and @shekhirin for reviews here
Comment on lines
+149
to
+163
| // We need to distinguish a full revert and a per-block revert. A full revert reverts | ||
| // changes starting at db tip all the way to a block. A per-block revert only reverts | ||
| // a block's changes. | ||
| // | ||
| // We need to calculate the full HashedPostState reverts for every block in the target | ||
| // range. The full HashedPostState revert for block N can be calculated as: | ||
| // | ||
| // | ||
| // ``` | ||
| // // where `extend` overwrites any shared keys | ||
| // state_revert(N) = state_revert(N + 1).extend(per_block_state_revert(N)) | ||
| // ``` | ||
| // | ||
| // We need per-block reverts to calculate the prefix set for each individual block. By using | ||
| // the per-block reverts to calculate full reverts on-the-fly we can save a bunch of memory. |
shekhirin
reviewed
Oct 1, 2025
yongkangc
reviewed
Oct 2, 2025
| /// | ||
| /// Handles Merkle trie changesets for storage and accounts. | ||
| #[value(name = "merkle-changesets")] | ||
| MerkleChangeSets, |
yongkangc
reviewed
Oct 2, 2025
yongkangc
reviewed
Oct 2, 2025
yongkangc
reviewed
Oct 2, 2025
yongkangc
reviewed
Oct 2, 2025
shekhirin
reviewed
Oct 2, 2025
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
shekhirin
reviewed
Oct 3, 2025
shekhirin
reviewed
Oct 3, 2025
shekhirin
reviewed
Oct 3, 2025
yongkangc
approved these changes
Oct 3, 2025
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18464
This implements a new sync stage for computing the Accounts/StoragesTrieChangeSets tables. Only changesets up to the finalized block (or 64 blocks ago, if no block is marked finalized) are generated, as these are the only ones necessary for the Engine API simplifications which this change is a part of.
Unwinding is implemented by simply clearing the changesets tables past the target block.