config: merge RemoveUntraceableHeaders into RemoveUntraceableBlocks #3922
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
Some of statesync module tests are failing due to #3795, will include this issue into this PR, we'll need to implement it anyway soon. |
roman-khimov
reviewed
Jun 3, 2025
AnnaShaleva
changed the title
Member
Author
|
@roman-khimov, it's still a draft, but you may review the idea of the last commit, we need to discuss this solution. An alternative (presize) solution messes up the code of statesync module. |
roman-khimov
reviewed
Jun 5, 2025
Member
roman-khimov
left a comment
roman-khimov
left a comment
There was a problem hiding this comment.
Looks like #3795 part of it is a bit premature.
Member
Author
|
Not yet ready for review, we still need to request trusted block to properly verify next headers. |
Member
Author
|
A lot of tests are still failing due to changes related to untraceable blocks removal, so still in the process of fixing them. The main logic is implemented and may be reviewed. |
AnnaShaleva
force-pushed
the
ruh
branch
11 times, most recently
from
10f918a to
802c342
Compare
Member
Author
|
@roman-khimov, let's perform some initial review. One problem marked with TODO is left and some new tests are still needed. All existing tests are finally fixed. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3922 +/- ##
==========================================
- Coverage 82.36% 82.31% -0.06%
==========================================
Files 345 347 +2
Lines 48773 48965 +192
==========================================
+ Hits 40173 40305 +132
- Misses 6924 6974 +50
- Partials 1676 1686 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
AnnaShaleva
force-pushed
the
ruh
branch
2 times, most recently
from
004e646 to
6cbafb5
Compare
1. Optimize it a bit by searching from the first suitable batch of headers at the DB Search level. 2. Rename it since this method removes the head of header hashes (we'll have code that removes tail of it soon). Signed-off-by: Anna Shaleva <[email protected]>
Early stop is not hard to support and it will be useful for untraceable header hashes removal. Signed-off-by: Anna Shaleva <[email protected]>
Use `cont` instead of `f` for processing function, otherwise it's not clear what is the return value meaning from the caller's side. Signed-off-by: Anna Shaleva <[email protected]>
Light nodes synchronized over NeoFSStateExchange won't have these data anyway, hence it would be nice to have unified behaviour for all light nodes. Old header hashes are not used anyway. Signed-off-by: Anna Shaleva <[email protected]>
AnnaShaleva
force-pushed
the
ruh
branch
5 times, most recently
from
f170b8f to
55a5669
Compare
AnnaShaleva
commented
Jul 2, 2025
Member
Author
|
@roman-khimov just for the record, GH tests are fixed (there was a bug actually, but it revealed only on slow machines), various manual node synchronization scenarios are checked, untraceable blocks removal check is in progress. Ready for review. |
Member
Author
|
Untraceable blocks removal on my machine with LevelDB and default GCP takes seconds (4-8 seconds) on syncing mainnet node. I'd say it's acceptable, especially comparing with MPT GC: |
roman-khimov
approved these changes
Jul 2, 2025
| } | ||
| if s.bc.BlockHeight() > p-2*s.syncInterval { | ||
| trustedH := s.bc.GetConfig().TrustedHeader.Index | ||
| if trustedH > s.bc.BlockHeight() { |
Member
There was a problem hiding this comment.
It's a pretty narrow case (s.bc.BlockHeight() > p-2*s.syncInterval), the node is almost in sync, so to me it should just work. I also doubt we can we end up here without changing TrustedHeight in the config (from the one originally used). Anyway, if the height is good and trusted (and local DB is trusted by definition) we better just keep going.
Signed-off-by: Anna Shaleva <[email protected]>
Signed-off-by: Anna Shaleva <[email protected]>
Should be a part of #2085, discovered during digging into the problem of inlined structures, ref. go-yaml/yaml#125. Signed-off-by: Anna Shaleva <[email protected]>
If batch of headers consists of stale headers only, then don't try to add any of them. Originally introduced by c3e73c5. Signed-off-by: Anna Shaleva <[email protected]>
Do not request the whole batch of headers if syncing via P2PStateExchangeExtensions or via NeoFSStateSyncExtensions. Instead, request headers starting from some trusted height. Introducing trusted header allows to enable tests related to untraceable blocks/headers removal. Close #3795. Signed-off-by: Anna Shaleva <[email protected]>
Use LE to log header hash. Signed-off-by: Anna Shaleva <[email protected]>
Signed-off-by: Anna Shaleva <[email protected]>
No functional changes. Signed-off-by: Anna Shaleva <[email protected]>
No functional changes. Signed-off-by: Anna Shaleva <[email protected]>
If corresponding header was stored in the DB we need to verify incoming block hash against this header. Otherwise, we may end up in a corrupted DB and malicious block saved to the database. Signed-off-by: Anna Shaleva <[email protected]>
When submitting block, we need to distinguish if it already exists on the chain or if it's a block from future. Signed-off-by: Anna Shaleva <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Close #3920, close #3795.
TODO:
Blocksmode).HeaderHashesinitialization on node restart after statesync interruption.