-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(restore): Introduce incremental restore #7942
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. Nice change. 🎉
One issue that we discussed regarding the stale predicates not getting deleted bcoz of predicate move.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also compare the current manifest with the last manifest that was applied, and drop any predicates that have moved etc.
Secondly, use the incremental stream writer.
Reviewed 9 of 9 files at r1.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ahsanbarkati)
worker/draft.go, line 677 at r1 (raw file):
defer closer.Done() glog.Infof("Got restore proposal at Index:%d, ReadTs:%d",
Index: <space> %d
worker/online_restore.go, line 245 at r1 (raw file):
} type IncrementalWriter struct {
Don't need this. We should ideally use incremental stream.
worker/online_restore.go, line 350 at r1 (raw file):
if mapRes.shouldDropAll { if err := pstore.DropAll(); err != nil { return errors.Wrap(err, "failed to reduce incremental restore map")
failed to drop all ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few nit picks. Rest looks good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 1 of 48 files reviewed, 4 unresolved discussions (waiting on @manishrjain and @NamanJain8)
worker/draft.go, line 677 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Index: <space> %d
Done.
worker/online_restore.go, line 245 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Don't need this. We should ideally use incremental stream.
Done.
worker/online_restore.go, line 350 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
failed to drop all ...
Done.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode. (cherry picked from commit dfa5dae)
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode. (cherry picked from commit dfa5dae)
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
This commit introduces incremental restore. It allows incremental backups to be restored on top of a set of already restored backups. In between two incremental restores, the cluster is in draining mode.
Incremental restore can be used to restore a set of incremental backups on a cluster with a part of the backup already restored.
No mutations should be made between a normal restore and an incremental restore.
This change is