feat: FCU unwind: properly reorg in-memory canonical state and update latest block#17938
Conversation
When a `forkchoiceUpdated` call points to a canonical ancestor of the current head (an unwind scenario), the engine's internal state for the "latest" block was not being reverted to match. This caused a state desynchronization where components like the transaction pool would operate on a stale, more advanced block state. This could lead to errors, such as "nonce too low", when validating new transactions against the incorrect (post-reorg) state. This commit introduces the `update_latest_block_to_canonical_ancestor` method, which is now called during FCU processing. This function ensures that both the `TreeState` and the `CanonicalInMemoryState` are correctly updated to the new head, resolving the state inconsistency after a reorg.
031e683 to
27c8e17
Compare
|
@mattsse I've rebased on the latest main branch changes. Could you please review? I'd recommend checking by commit since one commit addresses linting issues with |
|
Hi @pycckuu, can you leave out the linting changes which aren't related to the fix being submitted? They make it impossible to review this. |
I can definitely do that, but as mentioned earlier, reviewing commit-by-commit would be much more manageable. I'm also unsure how to handle linting issues that originate from the main branch and code I haven't modified. |
27c8e17 to
33672f4
Compare
|
@mediocregopher I've removed the styling commit. Could you please review it? Thanks in advance. |
mediocregopher
left a comment
There was a problem hiding this comment.
One nit and one question, will want @mattsse to look at it as well
| // Update the latest block state to reflect the canonical ancestor. | ||
| // This ensures that state providers and the transaction pool operate with | ||
| // the correct chain state after forkchoice update processing. | ||
| self.update_latest_block_to_canonical_ancestor(&canonical_header)?; |
There was a problem hiding this comment.
Is it correct that this is being called after the process_payload_attributes if-branch? If attrs.is_some() then this line never gets hit.
33672f4 to
b79881f
Compare
The previous implementation only updated the canonical head header on a forkchoice update. This was insufficient for handling reorgs where the new head is an ancestor of the current head (an unwind). This discrepancy could lead to a stale in-memory state. When the state provider attempted to access account information, it would fall back to the stale database state, causing transaction validation failures such as "nonce too low" errors. This commit refactors the logic to correctly handle unwinds. It now detects when the new head number is lower than the current one and treats it as a reorg. It collects the now-invalid blocks from the in-memory state and updates the chain by removing them and loading the new canonical ancestor. This ensures the in-memory state accurately reflects the canonical chain after a reorg.
b79881f to
15e077a
Compare
mediocregopher
left a comment
There was a problem hiding this comment.
lgtm, pending matt's review
|
gentle ping @mattsse! |
mattsse
left a comment
There was a problem hiding this comment.
I'm okay with this because this code path is only reachable if explicitly opt-in.
… latest block (paradigmxyz#17938) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
When PruneMode::Full is used for receipts, the pruner would prune all receipts up to the tip block immediately after persistence. This causes a race condition during chain reorganizations: 1. Block A at height N is persisted to disk 2. Pruner runs and prunes A's receipts 3. Block B at height N arrives (reorg) 4. FCU tries to make B canonical 5. on_new_head calls canonical_block_by_hash(A) to walk back old chain 6. canonical_block_by_hash tries to reconstruct ExecutedBlock from disk 7. get_state() fails with 'no receipt found' because receipts were pruned This fix adds MINIMUM_RECEIPTS_DISTANCE (64 blocks) to ensure receipts and bodies are retained long enough to handle any potential reorgs. Fixes a regression introduced by PR #17938 which added canonical_block_by_hash that assumes ExecutedBlock can always be reconstructed from disk.
Fix FCU when new head is an ancestor (unwind). Previously we only updated the header, leaving in-memory canonical state stale, causing tx validation issues (e.g., "nonce too low"). Now we detect unwind, reorg in-memory state (remove old blocks, load ancestor), and keep TreeState and CanonicalInMemoryState in sync.
Code Changes
log_chain_update_type,handle_chain_unwind,collect_blocks_for_removal,apply_canonical_ancestor_via_reorg,handle_chain_advance_or_same_height,ensure_block_in_memory.update_latest_block_to_canonical_ancestoradded, then superseded by full unwind handling.test_fcu_with_canonical_ancestor_updates_latest_blockasserting both TreeState and in-memory head match the ancestor after FCU.Reason for Changes
Correctness: prevent stale in-memory state on reorgs that led to txpool/state provider errors.
Impact of Changes
Test Plan
cargo test -p engine_tree test_fcu_with_canonical_ancestor_updates_latest_block.How the reviewer should test the fix
Other useful info
If ancestor block missing from storage, we warn and still update header to avoid desync.
Closes #17798