We need access to a worldstate image of the last 128 blocks with bonsai

[matkt]

With bonsai we have the worldstate of the head and tries logs to be able to rollback or rollforward the worldstate to an old/recent block. This technique with trielogs works very well in the case of reorgs but has limitations.

In particular with the snapsync as a server we must be able to provide the nodes of the worldstate of the last 128 blocks. If we have to rollback after every request it will be a lot of work for the node and risks impacting the stability of the node.

To avoid this it will be preferable to keep an image of the last 128 blocks and rocksdb's snapshot feature might help to do this

http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.html

This feature can also help make bonsai more stable. If we detect worldstate corruption for some reason (crash or bug). We could go back and take the worldstate of block head-1 and resume synchronization from this one

Example :

• 1 : the node tries to import block h+1
• 2 : stateroot mismatch or other issue detected
• 3 : besu take the worldstate image of block h-1 and try again to import block h
• 4: the database is fixed thanks to the new import
• 5: we import block h+1 again

[garyschulte]

As we discussed, a persistent snapshot (Checkpoint) seems like a viable source to rollback to whereas a collection of n (e.g. 128) in-memory snapshot-transaction-based worldstates is likely a more suitable mechanism for snapsync server.

see https://github.com/garyschulte/besu/tree/feature/bonsai-isolation (re-pushed and rebased off of main) and specifically:

RocksDbColumnarKeyValueStorage.takeSnapshot()
RocksDbColumnarKeyValueSnapshot

@jflo