Add Parquet Modular decryption (read) support + `encryption` flag#6637

Merged

alamb merged 99 commits intoapache:mainfrom

rok:decryption-basics-fork

Mar 12, 2025

Commits on Feb 17, 2025

first commit

ggershinsky
authored and
rok
committed
Use ParquetMetaDataReader
rok
committed
Fix CI
rok
committed
test
rok
committed
save progress
rok
committed
work
rok
committed
Review feedback
rok
committed
page decompression issue
rok
committed
add update_aad
rok
committed
Change encrypt and decrypt to return Results

adamreeve
authored and
rok
committed
Use correct page ordinal and module type in AADs

adamreeve
authored and
rok
committed
Tidy up ordinal types

adamreeve
authored and
rok
committed
Lint
rok
committed
Fix regular deserialization path
rok
committed
cleaning
rok
committed
Update data checks in test

adamreeve
authored and
rok
committed
start non-uniform decryption
rok
committed
Add missing doc comments

adamreeve
authored and
rok
committed
Make encryption an optional feature

adamreeve
authored and
rok
committed
Handle when a file is encrypted but encryption is disabled or no decryption properties are provided

adamreeve
authored and
rok
committed
Allow for plaintext footer
rok
committed
work
rok
committed
Fix method name

adamreeve
authored and
rok
committed
work
rok
committed
Minor
rok
committed
work
rok
committed
work
rok
committed
work
rok
committed
Fix reading to end of file

adamreeve
authored and
rok
committed
Refactor tests

adamreeve
authored and
rok
committed
Fix non-uniform encryption configuration

adamreeve
authored and
rok
committed
Don't use footer key for non-encrypted columns

adamreeve
authored and
rok
committed
Rebase and cleanup
rok
committed
Cleanup
rok
committed
Cleanup
rok
committed
Cleanup
rok
committed
Cleanup
rok
committed
Cleanup
rok
committed
Cleanup
rok
committed
lint
rok
committed
Remove encryption setup
rok
committed
Fix building with ring on wasm
rok
committed
file_decryptor into a seperate module
rok
committed
lint
rok
committed
FileDecryptionProperties should have at least one key
rok
committed
Move cyphertext reading into decryptor
rok
committed
More tidy up of footer key handling

adamreeve
authored and
rok
committed
Get column decryptors as RingGcmBlockDecryptor

adamreeve
authored and
rok
committed
Use Arc<dyn BlockDecryptor>

adamreeve
authored and
rok
committed
Fix file metadata tests

adamreeve
authored and
rok
committed
Handle reading plaintext footer files without decryption properties

adamreeve
authored and
rok
committed
Split up encryption modules further

adamreeve
authored and
rok
committed
Error instead of panic for AES-GCM-CTR

adamreeve
authored and
rok
committed
load_async
rok
committed
new_with_options
rok
committed
Add tests
rok
committed
get_metadata
rok
committed
Add CryptoContext to async_reader
rok
committed
Add row_group_ordinal to InMemoryRowGroup
rok
committed
Adjust docstrings
rok
committed
Apply suggestions from code review

rok
and
adamreeve
committed
Review feedback
rok
committed
move file_decryption_properties into ArrowReaderOptions
rok
committed
make create_page_aad method of CryptoContext
rok
committed
Review feedback
rok
committed
Infer ModuleType in create_page_aad
rok
committed
add create_page_header_aad
rok
committed
Review feedback
rok
committed
Update parquet/src/arrow/async_reader/store.rs

rok
and
etseidl
committed
Review feedback
rok
committed

Commits on Mar 6, 2025

Commits on Mar 8, 2025

Commits on Mar 10, 2025

Commits on Mar 11, 2025

Commits on Mar 12, 2025