Skip to content

frozen-abi: add imbl::GenericHashMap#583

Merged
2501babe merged 6 commits intoanza-xyz:masterfrom
2501babe:20260224_frozmap
Mar 4, 2026
Merged

frozen-abi: add imbl::GenericHashMap#583
2501babe merged 6 commits intoanza-xyz:masterfrom
2501babe:20260224_frozmap

Conversation

@2501babe
Copy link
Copy Markdown
Member

@2501babe 2501babe commented Feb 24, 2026

anza-xyz/agave#10762 intends to replace im with imbl as the backing store of StakesCache. the current crate is no longer maintained, while the new crate is a drop-in replacement under active development, with significant improvements

testing with my own benchmarks (anza-xyz/agave#10760) and confirmed by alessandro on mainnet, this change speeds up epoch cutover by 30-50%. however, we must implement AbiExample for imbl::HashMap to satify the frozen-abi contract on Bank. the way this is done for im is a simple impl in sdk

one of the challenges with copying this for imbl is the perf improvements we want are in imbl@v7.0.0, but this version depends on rust 1.85, while sdk enforces a minimum supported version of 1.81. the alternative is to put imbl::HashMap under a newtype wrapper and implement AbiExample in agave, with no sdk changes. this ends up being quite unpleasant because it requires us to write a different Serialize for the frozen-abi feature gate that returns some other type, because a wrapper over imbl::HashMap does not save us from needing AbiExample on every underlying type. aside from being ugly, i feel this violates the spirit of frozen-abi because we are explicitly lying to it about the serde representation of our type

"if only there were a better way!" there is. but i leave it to you to decide if i am a clever sort or if im guilty of Crate Crimes. this pr does the simple AbiExample for imbl::HashMap in sdk, identical to the treatment of im::HashMap. but i allow both imbl@v6 and imbl@7 in Cargo.toml, and give sdk a Cargo.lock enforcing 6. this means sdk is happy on rust 1.81. but the open version allows agave to use 7 without issue, getting us the perf we desire

none of this should affect the actual thing these tests care about, which is the snapshot format. im::HashMap and imbl::HashMap both go to the serde serialized representation of a platonic hashmap

@2501babe 2501babe self-assigned this Mar 3, 2026
Comment on lines +434 to +436
#[cfg(not(target_os = "solana"))]
impl<K, V, S, P> AbiExample for imbl::GenericHashMap<K, V, S, P>
where
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imbl::HashMap is just a type alias for this struct with two of the generic params held fixed

@2501babe 2501babe marked this pull request as ready for review March 3, 2026 06:02
@2501babe 2501babe requested a review from joncinque March 3, 2026 06:02
Copy link
Copy Markdown
Collaborator

@joncinque joncinque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hehe very clever! Confirming the types are equivalent in v6 and v7, let's just add a comment so we don't mess this up in the future

Comment thread Cargo.toml Outdated
hex = "0.4.3"
hmac = "0.12.1"
im = "15.1.0"
imbl = ">= 6.0.0, < 8.0.0"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just add a comment here so we don't mess this up in the future? Something like

# Use v6 in the SDK to maintain a lower rust-version, but allow v7 for Agave which uses a higher rust-version and needs v7 performance

Copy link
Copy Markdown
Member Author

@2501babe 2501babe Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

73b053c i put it to the right rather than above to copy the style of the serde comments

Copy link
Copy Markdown
Collaborator

@joncinque joncinque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@2501babe 2501babe merged commit 44059b0 into anza-xyz:master Mar 4, 2026
30 checks passed
@2501babe 2501babe deleted the 20260224_frozmap branch March 4, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants