Skip to content

perf(serde_snapshot): deserialize stakes in bank fields into Vec#9831

Merged
kskalski merged 1 commit intoanza-xyz:masterfrom
kskalski:ks/delegation_stakes
Jan 12, 2026
Merged

perf(serde_snapshot): deserialize stakes in bank fields into Vec#9831
kskalski merged 1 commit intoanza-xyz:masterfrom
kskalski:ks/delegation_stakes

Conversation

@kskalski
Copy link
Copy Markdown

@kskalski kskalski commented Jan 7, 2026

Problem

Building im::HashMap is quite slow and its creation for stakes maps dominates deserializing snaphsot, which is on critical path for startup. Bincode serialization format doesn't distinguish map from vector of (key, value) pairs, so we can instead quickly deserialize just a collection and then create final maps out of that, which may happen concurrently out of critical path or with use of rayon.

In fact currently stakes map is constructed by collecting deserialized delegations im hashmap back into vector (sic!) and running rayon fold to generate stakes map with account data (also fetches the account contents).

Summary of Changes

  • add DeserializableStakes<T> and DeserializableVersionedEpochStakes structs that use Vec instead of im::HashMap to store stakes (address, X) entries
  • never deserialize Stakes
  • deserialize bank fields using cheaper representation for stakes, which shorten time before we start accounts db initialization
  • move conversion of deserialized stakes in bank fields into real stakes maps after accounts db is initialized
  • in case of initialization from archive, this skips expensive deserialization that happens currently for both full and incremental snapshots, since we only run conversion after selecting one of the field sets

Note: this change opens up opportunity to parallelize the longest operation (building of epoch stakes im map) with accounts db initialization, not included in this PR as it's a bit larger refactor.

Performance change
  • Initializing from dir (fastboot) - speed up comes from building stakes delegation map, the epoch stakes cost is mostly shifted from beginning of bank_from_snapshot_dir to its later stage:

    • master - solana_runtime::serde_snapshot::deserialize_bank_fields 5.25s (before accountds db can run generate index) + solana_runtime::bank::Bank::new_from_snapshot 2.34s (after accountds db is initialized),
    • PR - solana_runtime::serde_snapshot::deserialize_bank_fields 1.4s + solana_runtime::bank::Bank::new_from_snapshot 5.2s
  • Initializing from archive - we skip one round of deserialize_bank_fields, since we only create expensive stakes maps for one of the snapshots instead of both (from full and incremental), so it brings some ~5s speedup

@kskalski kskalski marked this pull request as ready for review January 7, 2026 07:03
@brooksprumo brooksprumo removed the request for review from apfitzge January 7, 2026 13:13
@brooksprumo
Copy link
Copy Markdown

brooksprumo commented Jan 7, 2026

Andrew is out currently, so I've removed him as a reviewer.

@brooksprumo brooksprumo requested a review from jstarry January 7, 2026 14:21
Copy link
Copy Markdown

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice perf win!

@kskalski kskalski added this pull request to the merge queue Jan 12, 2026
Merged via the queue into anza-xyz:master with commit 8c873de Jan 12, 2026
47 checks passed
@kskalski kskalski deleted the ks/delegation_stakes branch January 12, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants