doc(restore): add docs for mutations in between incremental restores #8908
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mangalaman93
requested review from
meghalims,
sanjayk-github-dev,
harshil-goel,
all-seeing-code,
billprovince and
joshua-goldstein
as code owners
dgraph-bot
added
area/testing
mangalaman93
changed the title
harshil-goel
previously approved these changes
Jul 18, 2023
If we do mutations in between incremental restores, those will get lost. The next incremental restore will overwrite any data that is written in between two incremental restores.
mangalaman93
force-pushed
the
aman/incre-debug
branch
from
ccdb446 to
129b4e8
Compare
shivaji-kharse
approved these changes
Jul 18, 2023
harshil-goel
approved these changes
Jul 18, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If we do mutations in between incremental restores, those will get lost. The next incremental restore will overwrite any data that is written in between two incremental restores. It is not very useful to try to block the next incremental restore after in between mutations because it is really hard to detect that a change was made.
For example, let's say we detect the mutations done in between two incremental restores by looking at the data in memtable. But then, if we restart Dgraph, or drop any other attribute (predicate), that may end up doing compaction and would persist all the data from memtable to sstables. This would make it impossible to figure out that a change was made.
It wouldn't be very useful to store this information (that a mutation was done) in an in memory flag either. It would get destroyed if the alpha restarts. Any other methodology seems too complex to me. I think I'd rather keep the semantics of the incremental restore to say that it will overwrite any in between mutations that are done if any.
Closes: DGRAPHCORE-276
Docs: WIP