Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast pruner start #1756

Merged
merged 15 commits into from
Aug 29, 2023
Merged

Fast pruner start #1756

merged 15 commits into from
Aug 29, 2023

Conversation

Harrm
Copy link
Contributor

@Harrm Harrm commented Aug 23, 2023

Referenced issues

Description of the Change

Pruner starts immediately at a cost of no longer pruning the nodes already in storage at the time of the node startup (because without the long full state ref counting the pruner has no idea whether it can prune them or they are a part of the current state). CLI option --enable-thorough-pruning turns the old behaviour on (long startup, slightly better pruning).

Benefits

The node starts immediately.

Possible Drawbacks

Less effective pruning.

Alternate Designs

Asynchronous reference counting (and pruning overall) -- can be implemented in the future, but it's more complicated and with the reference counting taking half an hour on HDDs doesn't seem like a complete fix.

Persistent pruner state -- can be implemented in future, but it's more complicated and currently there's no clear vision how to do it.

@Harrm Harrm requested a review from iceseer August 28, 2023 11:34
@Harrm Harrm self-assigned this Aug 28, 2023
@Harrm Harrm marked this pull request as ready for review August 28, 2023 11:34
@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

Merging #1756 (a5d00b6) into master (604aef1) will decrease coverage by 0.02%.
The diff coverage is 4.00%.

❗ Current head a5d00b6 differs from pull request most recent head 30dd5df. Consider uploading reports for the commit 30dd5df to get more accurate results

@@            Coverage Diff             @@
##           master    #1756      +/-   ##
==========================================
- Coverage   21.71%   21.69%   -0.02%     
==========================================
  Files         746      746              
  Lines       32085    32100      +15     
  Branches    16661    16672      +11     
==========================================
- Hits         6966     6964       -2     
- Misses      19306    19318      +12     
- Partials     5813     5818       +5     
Files Changed Coverage Δ
core/application/app_configuration.hpp 100.00% <ø> (ø)
core/application/impl/app_configuration_impl.cpp 20.37% <0.00%> (-0.11%) ⬇️
core/application/impl/app_configuration_impl.hpp 22.82% <0.00%> (-0.51%) ⬇️
core/runtime/common/memory_allocator.cpp 53.52% <0.00%> (-0.65%) ⬇️
...e/storage/trie/impl/persistent_trie_batch_impl.cpp 26.66% <0.00%> (ø)
core/storage/trie_pruner/impl/trie_pruner_impl.hpp 66.66% <ø> (ø)
core/storage/trie_pruner/impl/trie_pruner_impl.cpp 29.54% <6.25%> (-1.56%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@Harrm Harrm merged commit 61c86c7 into master Aug 29, 2023
8 of 17 checks passed
@Harrm Harrm deleted the feature/fast-pruner-start branch August 29, 2023 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants