Continuous Trie Log Pruning#6075
Closed
siladu wants to merge 13 commits into
Closed
Conversation
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Separate out the concepts of world state caching from trie log management Make AbstractTrieLogManager a concrete implemenation (to be further renamed/refactored next commit) Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Feature toggled by --Xtrie-log-pruning-enabled Each time a trie log is persisted, the current trie log is cached and the pruner is run against the oldest entries in the cache. This makes no attempt to manage the backlog of old trie logs, it will only prune what has been added to the cache, i.e. trie logs that have been added since the feature was enabled. Pruner limit exists in case of exceptional circumstances, but we should only ever be pruning all the forks for a single block number during each prune execution. Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Was only used for supporting test code and can instead reuse static factory from InMemoryKeyValueStorageProvider Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
|
siladu
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| final BonsaiWorldStateUpdateAccumulator localUpdater, | ||
| final Hash forWorldStateRootHash, | ||
| final BlockHeader forBlockHeader, | ||
| final BonsaiWorldState forWorldState) { |
Check warning
Code scanning / CodeQL
Non-synchronized override of synchronized method
siladu
added
TeamGroot
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Inialised once on startup Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Contributor
Author
|
Abandoning this approach in favour of an adapted form of the original #6026 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--Xtrie-log-pruning-enabledTrieLogManager.saveTrieLogThis is a second take of #6026
Currently built on #6072 (diff: siladu/besu@refactor-trie-log-manager...trie-log-pruning-take2)
One downside of this PR compared to #6026 is that restarting besu may leave a gap in the pruned trie logs because we don't preload the cache. This will have to be considered as part of managing the backlog. It may mean that we can't simply rely on a "one-off" resync or prune subcommand. However, these un-pruneable trie logs should be negligible compared to the saving gained for long running nodes.