eth: reset skeleton after chain rewinded

[jsvisa]

When I'm rewinding my chain to a old header, geth failed to importing the new blocks, log as below:

INFO [06-26|15:17:13.835] Rewound to block with state              number=682,101 hash=55b54a..d1afb1
INFO [06-26|15:17:13.836] Rewound to block with state              number=682,100 hash=ad9ac3..07566c
ERROR[06-26|15:17:13.837] Header not found                         number=682,115 hash=a353e4..ae3882
ERROR[06-26|15:17:13.837] Header not found                         number=682,115 hash=a353e4..ae3882
ERROR[06-26|15:17:13.837] Chain view: block hash unavailable       number=40019   head=682,115
ERROR[06-26|15:17:13.837] Log index head rendering failed, reverting to unindexed mode error="failed to advance log iterator at 233891 while rendering map 3: receipts not found for block 40019"
INFO [06-26|15:17:13.839] Loaded most recent local block           number=682,100 hash=ad9ac3..07566c age=3m1s
INFO [06-26|15:17:13.839] Loaded most recent local finalized block number=682,046 hash=6185d6..0b667b age=14m37s
INFO [06-26|15:17:13.839] Loaded last snap-sync pivot marker       number=682,044
INFO [06-26|15:17:20.238] Generating state snapshot                root=62b2ec..6dc787 in=4dd531..a66e82 at=34e5fe..301445 accounts=4,084,718       slots=1,311,915      storage=297.62MiB  dangling=0 elapsed=41.962s    eta=1m36.057s
WARN [06-26|15:17:24.870] Ignoring payload with missing parent     number=682,116 hash=f98a00..b7d677 parent=a353e4..ae3882 reason="chain gapped, head: 682112, newHead: 682116"
INFO [06-26|15:17:25.140] Forkchoice requested sync to new head    number=682,116 hash=f98a00..b7d677 finalized=unknown
INFO [06-26|15:17:25.140] Restarting sync cycle                    reason="chain gapped, head: 682112, newHead: 682116"
INFO [06-26|15:17:27.796] Syncing beacon headers                   downloaded=15 left=0 eta=0s
INFO [06-26|15:17:28.238] Generating state snapshot                root=62b2ec..6dc787 at=614675..4edb41 accounts=5,104,795       slots=1,439,343      storage=354.46MiB  dangling=0 elapsed=49.962s    eta=1m21.525s
INFO [06-26|15:17:29.416] Syncing beacon headers                   downloaded=27 left=0 eta=0s
INFO [06-26|15:17:30.411] Syncing beacon headers                   downloaded=39 left=0 eta=0s
INFO [06-26|15:17:31.052] Syncing beacon headers                   downloaded=51 left=0 eta=0s
INFO [06-26|15:17:31.651] Syncing beacon headers                   downloaded=63 left=0 eta=0s
INFO [06-26|15:17:32.194] Syncing beacon headers                   downloaded=75 left=0 eta=0s
INFO [06-26|15:17:32.740] Syncing beacon headers                   downloaded=87 left=0 eta=0s
INFO [06-26|15:17:33.269] Syncing beacon headers                   downloaded=99 left=0 eta=0s
INFO [06-26|15:17:33.887] Syncing beacon headers                   downloaded=111 left=0 eta=0s
INFO [06-26|15:17:34.437] Syncing beacon headers                   downloaded=123 left=0 eta=0s
INFO [06-26|15:17:34.885] Syncing beacon headers                   downloaded=135 left=0 eta=0s
INFO [06-26|15:17:35.292] Syncing beacon headers                   downloaded=147 left=0 eta=0s

It seems that geth stucked at Syncing beacon headers, and later after I restarted geth, it continued to sync and import blocks.

After some investigate, I think it is because after we rewinding the chain, the skeleton syncer's subchain state and the rewinded state are not matched. As a result, it will keep trying to sync headers that don't match the new chain, causing the repeating logs.

So here we simplify reset the skeleton to the fresh state to fix the data inconsistent.

BTW, I'm not sure if this fix was related to #32098

[rjl493456442]

Can you provide log-level (e.g. DEBUG level) logs after resetting the head?

It sounds reasonable to me to reset the skeleton chain, just want to make sure what happens there.

Also, it will be very useful by adding a unit test.

[s1na]

I believe this was fixed as part of #33481. Closing.