Core: add key_metadata to ManifestFile spec #2675
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jackye1995
commented
Jun 4, 2021
| .add("deleted_data_files_count", deletedFilesCount) | ||
| .add("deleted_rows_count", deletedRowsCount) | ||
| .add("partitions", partitions) | ||
| .add("key_metadata", keyMetadata == null ? "null" : "(redacted)") |
Contributor
Author
There was a problem hiding this comment.
I would actually be more against using this approach to support redact, for 2 reasons:
- this is only for
toString. If you really want to see the details of the key metadata you can read through the manifest table, and that should show the information fully if you can decrypt an encrypted manifest list. I think that provides better access control. ChangingtoStringfeels to me like opening a backdoor. - if we can encrypt a manifest, we should be able to support redact configuration natively through encryption manager as a part of data masking. When we can do that, you don't need a forked version to use a custom static method for this purpose.
RussellSpitzer
approved these changes
Jun 11, 2021
Member
RussellSpitzer
left a comment
RussellSpitzer
left a comment
There was a problem hiding this comment.
This looks good to me, do we have anyone else we are waiting on to get this merged? @aokolnychyi ?
Contributor
Author
|
@RussellSpitzer we are not waiting for anyone to merge, but I thought for any format v2 change @rdblue should take a look and approve. If it's not necessary we will merge to unblock other subsequent changes. |
rdblue
reviewed
Jun 14, 2021
| } | ||
|
|
||
| @Override | ||
| public ByteBuffer keyMetadata() { |
Contributor
There was a problem hiding this comment.
Is this used in tests? We could just allow this to use the default implementation, unless we added tests that exercise it.
rdblue
approved these changes
Jun 14, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@flyrain @rdblue @ggershinsky
first step based on draft #2520, simply add
keyMetadatato the interface and implementations, pass innullfor any caller.