Skip to content

v1.18: blockstore: account for blockstore cleanup during shred insertion (backport of #1259)#1279

Merged
yihau merged 1 commit intov1.18from
mergify/bp/v1.18/pr-1259
May 10, 2024
Merged

v1.18: blockstore: account for blockstore cleanup during shred insertion (backport of #1259)#1279
yihau merged 1 commit intov1.18from
mergify/bp/v1.18/pr-1259

Conversation

@mergify
Copy link
Copy Markdown

@mergify mergify Bot commented May 10, 2024

v1.18 introduces many new expect and unwraps on blockstore invariants.

An audit shows that some of these invariants might not hold under extreme situations. Specifically any check relying on receiving a shred from get_shred_from_just_inserted_or_db may fail if the slot is cleaned up during shred insertion:

  1. Shred for slot S is received and compared successfully against blockstore.max_root() in shred fetch stage
  2. Shred enters blockstore shred insertion, insertion lock is grabbed
  3. Replay freezes and votes on a block, updating blockstore.max_root() past S
  4. Blockstore cleanup purges all slot columns up to this new blockstore.max_root(), including the columns for the shred currently being inserted (insertion lock is not checked here)
  5. Shred insertion panics as the shred expected from get_shred_from_just_inserted_or_db is no longer present.

(4) is unlikely to happen as @steviez pointed out here #1151 (comment), but for safety this should still be accounted for.


This is an automatic backport of pull request #1259 done by Mergify.

@mergify mergify Bot requested a review from a team as a code owner May 10, 2024 01:13
@mergify mergify Bot requested a review from a team May 10, 2024 01:13
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 10, 2024

Codecov Report

❌ Patch coverage is 25.00000% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.6%. Comparing base (be4a672) to head (e810e27).
⚠️ Report is 58 commits behind head on v1.18.

Additional details and impacted files
@@            Coverage Diff            @@
##            v1.18    #1279     +/-   ##
=========================================
- Coverage    81.6%    81.6%   -0.1%     
=========================================
  Files         827      827             
  Lines      225421   225430      +9     
=========================================
+ Hits       184095   184100      +5     
- Misses      41326    41330      +4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@steviez steviez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in favor of this change for BP - this PR is demoting several panics to error logs in the name of being cautious. No functional changes and again, just being VERY cautious for a situation that should never occur given other system parameters

@yihau yihau merged commit 75cc70d into v1.18 May 10, 2024
@yihau yihau deleted the mergify/bp/v1.18/pr-1259 branch May 10, 2024 17:11
anwayde pushed a commit to firedancer-io/agave that referenced this pull request Jul 23, 2024
…ion (backport of anza-xyz#1259) (anza-xyz#1279)

blockstore: account for blockstore cleanup during shred insertion (anza-xyz#1259)

(cherry picked from commit b5c5bd3)

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants