This repository was archived by the owner on Jan 22, 2025. It is now read-only.
Fix ref-count for multiple stores to the same pubkey in a slot, fixes zero lamport purge detection#12462
Merged
carllin merged 4 commits intosolana-labs:masterfrom Sep 26, 2020
Merged
Conversation
fa8f897 to
92401fe
Compare
Codecov Report
@@ Coverage Diff @@
## master #12462 +/- ##
=======================================
Coverage 82.0% 82.0%
=======================================
Files 354 354
Lines 82719 82732 +13
=======================================
+ Hits 67896 67913 +17
+ Misses 14823 14819 -4 |
ryoqun
reviewed
Sep 25, 2020
ryoqun
reviewed
Sep 25, 2020
ryoqun
reviewed
Sep 25, 2020
| .collect::<Vec<Pubkey>>(), | ||
| ) | ||
| }) | ||
| .collect() |
Contributor
There was a problem hiding this comment.
how about reduce()-ing here like this, which will be done in parallel by rayon?:
let pubkeys: HashSet<(Slot, Pubkey)> =
...
.reduce(|| HashSet<_>::new(), |mut reduced, pubkeys| reduced.extend(pubkeys)so that we can remove the need to construct cleaned_slot_keys later?
ryoqun
previously approved these changes
Sep 25, 2020
Contributor
There was a problem hiding this comment.
LGTM with nits, awesome bug fixing. I guess you found this by carefully reading the code to fight with the snapshot bloat issue at #12194 .
I guess this bug doesn't manifest as clear errors. Only this causes is dangling appendvecs, which doesn't reduce the snapshot size as expected.
Pull request has been modified.
Contributor
Author
|
@ryoqun I actually found it while writing those snapshot hash mismatch tests :) |
a2efc21 to
4c21ec7
Compare
4c21ec7 to
59d27eb
Compare
59d27eb to
ecec28a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Storing to the same key multiple times in a slot only keeps one entry alive for that (pubkey, slot) pair in the AccountsIndex, but increments the ref count multiple times. This means the zero-lamport purge logic will not accurately detect if an account can be purged
Summary of Changes
Only increment ref count once per store per pubkey/slot.
Fixes #