perf(serde_snapshot): deserialize stakes in bank fields into Vec#9831
Merged
kskalski merged 1 commit intoanza-xyz:masterfrom Jan 12, 2026
Merged
perf(serde_snapshot): deserialize stakes in bank fields into Vec#9831kskalski merged 1 commit intoanza-xyz:masterfrom
kskalski merged 1 commit intoanza-xyz:masterfrom
Conversation
|
Andrew is out currently, so I've removed him as a reviewer. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Building
im::HashMapis quite slow and its creation for stakes maps dominates deserializing snaphsot, which is on critical path for startup. Bincode serialization format doesn't distinguish map from vector of(key, value)pairs, so we can instead quickly deserialize just a collection and then create final maps out of that, which may happen concurrently out of critical path or with use of rayon.In fact currently stakes map is constructed by collecting deserialized delegations im hashmap back into vector (sic!) and running rayon fold to generate stakes map with account data (also fetches the account contents).
Summary of Changes
DeserializableStakes<T>andDeserializableVersionedEpochStakesstructs that useVecinstead ofim::HashMapto store stakes (address, X) entriesStakesNote: this change opens up opportunity to parallelize the longest operation (building of epoch stakes
immap) with accounts db initialization, not included in this PR as it's a bit larger refactor.Performance change
Initializing from dir (fastboot) - speed up comes from building stakes delegation map, the epoch stakes cost is mostly shifted from beginning of
bank_from_snapshot_dirto its later stage:solana_runtime::serde_snapshot::deserialize_bank_fields5.25s (before accountds db can run generate index) +solana_runtime::bank::Bank::new_from_snapshot2.34s (after accountds db is initialized),solana_runtime::serde_snapshot::deserialize_bank_fields1.4s +solana_runtime::bank::Bank::new_from_snapshot5.2sInitializing from archive - we skip one round of
deserialize_bank_fields, since we only create expensive stakes maps for one of the snapshots instead of both (from full and incremental), so it brings some ~5s speedup