Skip to content

feat(pruner, storage): prune receipts & save checkpoints to database#3733

Merged
shekhirin merged 22 commits intomainfrom
alexey/pruner-receipts
Jul 24, 2023
Merged

feat(pruner, storage): prune receipts & save checkpoints to database#3733
shekhirin merged 22 commits intomainfrom
alexey/pruner-receipts

Conversation

@shekhirin
Copy link
Member

@shekhirin shekhirin commented Jul 12, 2023

Resolves #3697

Adds pruning of receipts the the Pruner which is activated during the live sync. Pruning of receipts is done in one go, and the progress is printed every 10k receipts. After pruning is done, writes checkpoints to the database.

@shekhirin shekhirin added C-enhancement New feature or request A-db Related to the database A-pruning Related to pruning or full node labels Jul 12, 2023
@shekhirin shekhirin force-pushed the alexey/pruner-receipts branch from 4c0b0c5 to bd644cd Compare July 12, 2023 13:53
@shekhirin shekhirin force-pushed the alexey/pruner-receipts branch from 72f0db0 to 6278abb Compare July 12, 2023 14:29
@shekhirin shekhirin force-pushed the alexey/pruner-receipts branch from 6278abb to 7771c93 Compare July 12, 2023 14:32
@shekhirin shekhirin force-pushed the alexey/pruner-receipts branch from 5ff7dc6 to b46518b Compare July 12, 2023 15:37
@codecov
Copy link

codecov bot commented Jul 12, 2023

Codecov Report

Merging #3733 (12582e4) into main (cb0947d) will increase coverage by 0.02%.
The diff coverage is 74.50%.

Impacted file tree graph

Impacted Files Coverage Δ
bin/reth/src/debug_cmd/merkle.rs 1.14% <0.00%> (ø)
bin/reth/src/node/mod.rs 9.94% <0.00%> (-0.12%) ⬇️
bin/reth/src/stage/dump/merkle.rs 0.00% <0.00%> (ø)
crates/config/src/config.rs 64.65% <0.00%> (ø)
crates/primitives/src/lib.rs 100.00% <ø> (ø)
crates/primitives/src/prune/checkpoint.rs 100.00% <ø> (ø)
crates/prune/src/error.rs 0.00% <ø> (ø)
crates/storage/db/src/tables/mod.rs 53.65% <ø> (ø)
...tes/storage/provider/src/providers/database/mod.rs 30.88% <0.00%> (-0.35%) ⬇️
crates/storage/provider/src/providers/mod.rs 15.68% <0.00%> (-0.11%) ⬇️
... and 10 more

... and 12 files with indirect coverage changes

Flag Coverage Δ
integration-tests 15.57% <0.00%> (-0.05%) ⬇️
unit-tests 64.44% <74.50%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 26.44% <0.00%> (-0.04%) ⬇️
blockchain tree 83.02% <ø> (ø)
pipeline 89.62% <100.00%> (+0.02%) ⬆️
storage (db) 74.24% <61.11%> (-0.03%) ⬇️
trie 94.65% <ø> (ø)
txpool 46.00% <ø> (-0.57%) ⬇️
networking 77.72% <ø> (+0.04%) ⬆️
rpc 58.22% <ø> (-0.01%) ⬇️
consensus 65.32% <100.00%> (+0.07%) ⬆️
revm 33.68% <ø> (ø)
payload builder 6.61% <ø> (ø)
primitives 88.09% <36.36%> (+0.05%) ⬆️

@shekhirin shekhirin marked this pull request as ready for review July 17, 2023 06:51
@shekhirin shekhirin requested a review from gakonst as a code owner July 17, 2023 06:51
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks very clean and was easy to follow.

tests also look great

storage_history
/// Returns block up to which pruning needs to be done, inclusive, according to the provided
/// prune mode and tip.
pub fn prune_to_block(&self, mode: &PruneMode, tip: BlockNumber) -> BlockNumber {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe prune_target_block?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow-up PR will change the name

@joshieDo joshieDo enabled auto-merge July 19, 2023 16:24
@shekhirin shekhirin disabled auto-merge July 23, 2023 22:01
@shekhirin shekhirin enabled auto-merge July 24, 2023 15:26
@shekhirin shekhirin added this pull request to the merge queue Jul 24, 2023
@shekhirin shekhirin removed this pull request from the merge queue due to a manual request Jul 24, 2023
@shekhirin shekhirin enabled auto-merge July 24, 2023 16:15
@shekhirin shekhirin added this pull request to the merge queue Jul 24, 2023
Merged via the queue into main with commit 1ca7f3a Jul 24, 2023
@shekhirin shekhirin deleted the alexey/pruner-receipts branch July 24, 2023 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-db Related to the database A-pruning Related to pruning or full node C-enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Respect Receipts prune part in the pruner

3 participants