Skip to content

fix(pruner): history indices cursor.prev() emptiness check#4301

Merged
shekhirin merged 2 commits intomainfrom
alexey/fix-pruner-history-indices
Aug 22, 2023
Merged

fix(pruner): history indices cursor.prev() emptiness check#4301
shekhirin merged 2 commits intomainfrom
alexey/fix-pruner-history-indices

Conversation

@shekhirin
Copy link
Member

@shekhirin shekhirin commented Aug 21, 2023

Problem

First row in AccountHistory and StorageHistory tables wasn't pruned, but instead the row after it was. The issue is in this snippet:

if let Some(prev_value) = cursor
.prev()?
.filter(|(prev_key, _)| key_matches(prev_key, &key))
.map(|(_, prev_value)| prev_value)
{
// If current shard is the last shard for the sharded key that has
// previous shards, replace it with the previous shard.
cursor.delete_current()?;
// Upsert will replace the last shard for this sharded key with the
// previous value.
cursor.upsert(key.clone(), prev_value)?;
} else {
// If there's no previous shard for this sharded key,
// just delete last shard completely.
// Jump back to the original last shard.
cursor.next()?;
// Delete shard.
cursor.delete_current()?;
}

When we move a cursor to the previous row, it can be the first row of the table, so the cursor will not be actually moved and the cursor.prev() will return None. We assumed that we always move it, so the next movement forward via cursor.next() was actually going forward too much.

Solution

Check that previous row wasn't empty and we actually moved the cursor:

if prev_row.is_some() {
cursor.next()?;
}

Also, save on a no-op seek_exact if we were already at the last shard:

// Jump to the last shard for this key, if current key isn't already the last shard.
if key.as_ref().highest_block_number != u64::MAX {
cursor.seek_exact(last_key(&key))?;
}

The rest is just reformatting of the comments.

@shekhirin shekhirin marked this pull request as ready for review August 21, 2023 21:11
@shekhirin shekhirin requested a review from gakonst as a code owner August 21, 2023 21:11
@shekhirin shekhirin removed the request for review from gakonst August 21, 2023 21:11
@shekhirin shekhirin changed the title fix(pruner): history indices cursor.prev() empty check fix(pruner): history indices cursor.prev() empty check Aug 21, 2023
@codecov
Copy link

codecov bot commented Aug 21, 2023

Codecov Report

Merging #4301 (4a5d8d2) into main (34b68de) will increase coverage by 0.03%.
The diff coverage is 37.50%.

Impacted file tree graph

Files Changed Coverage Δ
crates/prune/src/pruner.rs 78.63% <37.50%> (-0.08%) ⬇️

... and 9 files with indirect coverage changes

Flag Coverage Δ
integration-tests 16.84% <0.00%> (+<0.01%) ⬆️
unit-tests 63.80% <37.50%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 26.12% <ø> (ø)
blockchain tree 82.56% <ø> (ø)
pipeline 90.07% <ø> (ø)
storage (db) 74.77% <ø> (ø)
trie 94.81% <ø> (ø)
txpool 48.22% <ø> (ø)
networking 77.71% <ø> (+0.16%) ⬆️
rpc 58.81% <ø> (-0.01%) ⬇️
consensus 63.53% <ø> (ø)
revm 32.01% <ø> (ø)
payload builder 6.82% <ø> (ø)
primitives 86.17% <ø> (+0.02%) ⬆️

@shekhirin shekhirin added C-bug An unexpected or incorrect behavior A-pruning Related to pruning or full node labels Aug 21, 2023
@shekhirin shekhirin changed the title fix(pruner): history indices cursor.prev() empty check fix(pruner): history indices cursor.prev() emptiness check Aug 21, 2023
Copy link
Collaborator

@joshieDo joshieDo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@shekhirin shekhirin added this pull request to the merge queue Aug 22, 2023
Merged via the queue into main with commit b78e10f Aug 22, 2023
@shekhirin shekhirin deleted the alexey/fix-pruner-history-indices branch August 22, 2023 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-pruning Related to pruning or full node C-bug An unexpected or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants