Manage file key by the file to encrypt itself - all changes #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: There are some comments on subclasses in EncryptedEnv module which are duplicate to their parent classes, it would be nice to remove the duplication and keep the consistency if the comments on parent classes updated in someday. Pull Request resolved: facebook#11549 Reviewed By: akankshamahajan15 Differential Revision: D47007061 Pulled By: ajkr fbshipit-source-id: 8bfdaf9f2418a24ca951c30bb88e90ac861d9016
apache/incubator-pegasus#1575 Cherry-pick from tikv@113b363 Summary: Introduce `KeyManagedEncryptedEnv` which wraps around `EncryptedEnv` but provides an `KeyManager` API to enable key management per file. Also implements `AESBlockCipher` with OpenSSL. Test Plan: not tested yet. will update. Signed-off-by: Yi Wu <[email protected]> Signed-off-by: tabokie <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@3d44a33 Summary: Instead of using openssl's raw `AES_encrypt` and `AES_decrypt` API, which is a low level call to encrypt or decrypt exact one block (16 bytes), we change to use the `EVP_*` API. The former is deprecated, and will use the default C implementation without AES-NI support. Also the EVP API is capable of handing CTR mode on its own. Test Plan: will add tests Signed-off-by: Yi Wu <[email protected]> --------- Signed-off-by: Yi Wu <[email protected]> Co-authored-by: yiwu-arbug <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@2360562 Summary: Fix NewRandomRWFile and ReuseWritableFile misuse of `GetFile()` and `NewFile()`. See inline comments. Test Plan: manual test with tikv Signed-off-by: Yi Wu <[email protected]> Co-authored-by: yiwu-arbug <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@93e89a5 fix bug: tikv/tikv#9115 Summary: we need to update encryption metadata via encryption::DataKeyManager, which cannot combine with the actual file operation into one atomic operation. In RenameFile, when the src_file has been removed, power is off, then we may lost the file info of src_file next restart. Signed-off-by: Xintao [[email protected]](mailto:[email protected]) Signed-off-by: Xintao <[email protected]> Co-authored-by: Xintao <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@bbd27cf used LinkFile instead of RenameFile api of key manager. But LinkFile needs check the dst file information, in RenameFile logic, we don't care about that. So just skip encryption for current file. Signed-off-by: Xintao [[email protected]](mailto:[email protected])
…cebook#9) apache/incubator-pegasus#1575 Cherry-pick from tikv@1868d12 Signed-off-by: Xintao <[email protected]> Signed-off-by: tabokie <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@4cebfc1 * Add SM4-CTR encryption algorithm * Adjust block size for sm4 encryption * Add UT for SM4 encryption * Adjust macros indentation for sm4 * Fix format for adding sm4 Signed-off-by: Jarvis Zheng <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@9464766 In some env, user installed openssl by yum install, and the openssl software may compiled with OPENSSL_NO_SM4 flag, so although the version is >= 1.1.1, but we still could not use sm4 in that situation. Signed-off-by: Jarvis Zheng <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@acc624f * hook delete dir in encrypted env * add a comment Signed-off-by: tabokie <[email protected]> Co-authored-by: Xinye Tao <[email protected]>
apache/incubator-pegasus#1575 Cherry-pick from tikv@14f36f8 (without compaction related code) * fix renaming encrypted directory Signed-off-by: tabokie <[email protected]>
acelyc111
changed the title
acelyc111
added a commit
that referenced
this pull request
Sep 15, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.