Skip to content

feat(pruner): percentage progress and prune only if key exists#3932

Merged
onbjerg merged 4 commits intomainfrom
alexey/pruner-batch-percentage
Jul 26, 2023
Merged

feat(pruner): percentage progress and prune only if key exists#3932
onbjerg merged 4 commits intomainfrom
alexey/pruner-batch-percentage

Conversation

@shekhirin
Copy link
Member

@shekhirin shekhirin commented Jul 26, 2023

  1. Log percentage progress for pruning:
2023-07-26T16:20:17.524869Z TRACE prune_transaction_lookup{to_block=3968777 prune_mode=Full}: pruner: Pruned transaction lookup entries=10000 progress="31.7%"
2023-07-26T16:20:17.559487Z TRACE prune_transaction_lookup{to_block=3968777 prune_mode=Full}: pruner: Pruned transaction lookup entries=10000 progress="31.8%"
  1. Delete the entry only if it exists, otherwise do nothing. Previously, we were crashing. It shouldn't happen during normal operation, but can happen during manual debugging.

@shekhirin shekhirin marked this pull request as draft July 26, 2023 15:42
@codecov
Copy link

codecov bot commented Jul 26, 2023

Codecov Report

Merging #3932 (f02f360) into main (d5ea168) will decrease coverage by 0.02%.
Report is 2 commits behind head on main.
The diff coverage is 35.71%.

Impacted file tree graph

Files Changed Coverage Δ
crates/prune/src/pruner.rs 88.99% <20.00%> (-1.34%) ⬇️
...torage/provider/src/providers/database/provider.rs 78.33% <75.00%> (+0.69%) ⬆️

... and 8 files with indirect coverage changes

Flag Coverage Δ
integration-tests 15.53% <0.00%> (+0.05%) ⬆️
unit-tests 64.46% <35.71%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 27.24% <ø> (ø)
blockchain tree 83.01% <ø> (ø)
pipeline 89.68% <ø> (ø)
storage (db) 74.28% <75.00%> (+0.09%) ⬆️
trie 94.70% <ø> (ø)
txpool 46.00% <ø> (-0.60%) ⬇️
networking 77.66% <ø> (-0.05%) ⬇️
rpc 58.55% <ø> (+0.16%) ⬆️
consensus 64.46% <ø> (ø)
revm 33.68% <ø> (ø)
payload builder 6.61% <ø> (ø)
primitives 88.06% <ø> (+0.01%) ⬆️

@shekhirin shekhirin marked this pull request as ready for review July 26, 2023 16:25
@shekhirin shekhirin requested review from joshieDo and mattsse July 26, 2023 16:25
@onbjerg onbjerg added C-enhancement New feature or request A-observability Related to tracing, metrics, logs and other observability tools A-pruning Related to pruning or full node C-bug An unexpected or incorrect behavior and removed C-enhancement New feature or request labels Jul 26, 2023
@onbjerg onbjerg added this pull request to the merge queue Jul 26, 2023
Merged via the queue into main with commit 0cfc424 Jul 26, 2023
@onbjerg onbjerg deleted the alexey/pruner-batch-percentage branch July 26, 2023 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-observability Related to tracing, metrics, logs and other observability tools A-pruning Related to pruning or full node C-bug An unexpected or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants