feat(clp): Postpone GlobalMetadataDB updates until archive writing finishes (fixes #685). #705
+86
−146
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
See @685 for high level PR goals.
re: implementation
Removed all writes during serialization to Global metadata DB.
When the archive serialization is complete, I do two things.
(1) I add the archive metadata to the global metadata DB.
(2) I copy all the file metadata from the archive-level metadataDB to the global metadataDB.
re: I explored other methods such as maintaining a list of files in the archive writer class, and adding files to the list as they are compressed. Then at the end, i could add the list to global DB. However, I thought this could be more prone to bugs if someone adds a new method to compress files, or if the file metadata is altered at a later point. Moreover, we are storing the data twice, once in archive writer and once in local DB. Therefore, I thought a simple solution was just to query the local DB at the end, and copy over the metadata.
Validation performed
TBD - draft