Add pre-migration step to migrate script

[alecps]

The rsync command currently used in the celo-migrate script can be run as part of the pre-migration step, without applying any transformations or state changes on top of it. It seems the --update and --delete flags can be used to only rsync new data during the actual migration.

Tasks

Beta Give feedback

1. Add pre-migrate sub-command
2. Have rsync run as part of pre-migration with proper flags
3. Consider adding back state-dry-run functionality

[piersy]

Hey @alecps what's the reason for running this separately in a pre-migrate step?

[alecps]

@piersy the reason is that the full migration will apply transformations to the newDb (as part of the non-ancients and state migration steps). These transformations will affect what the rsync command detects as differences between the old and new dbs the second time it is run, causing it to copy over the wrong data and corrupt the newDb. So, we want a way to run only the ancients migration and rsync commands (the only steps that can be paused and resumed) together ahead of time in a "pre-migration". The pre-migration step is still part of the full migration, where it will only apply to any newly frozen blocks or other updates in oldDB since the last pre-migration was run.

Open to other ideas on how to structure this if you have any!