-
Notifications
You must be signed in to change notification settings - Fork 1.5k
PARQUET-2022: ZstdDecompressorStream should close zstdInputStream
#889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
`ZstdDecompressorStream` should close its resource because `CompressionInputStream.close` closes only the inner stream.
```
public class ZstdDecompressorStream extends CompressionInputStream {
private ZstdInputStream zstdInputStream;
public ZstdDecompressorStream(InputStream stream) throws IOException {
super(stream);
zstdInputStream = new ZstdInputStream(stream);
}
}
```
Fokko
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks for fixing this @dongjoon-hyun
|
Thank you, @Fokko ! |
|
Could you review this, @gszadovszky ? |
|
Could you review this, @ggershinsky ? |
|
will do |
|
Thank you so much, @ggershinsky ! |
ggershinsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
Thank you for your review and approval, @ggershinsky ! |
|
Gentle ping, @gszadovszky . |
|
Thank you, @Fokko , @ggershinsky , @gszadovszky ! |
|
Thanks @dongjoon-hyun for working on this! cc @vectorijk |
|
Thank you, @shangxinli . |
* 'master' of https://github.com/apache/parquet-mr: (222 commits) PARQUET-2052: Integer overflow when writing huge binary using dictionary encoding (apache#910) PARQUET-2041: Add zstd to `parquet.compression` description of ParquetOutputFormat Javadoc (apache#899) PARQUET-2050: Expose repetition & definition level from ColumnIO (apache#908) PARQUET-1761: Lower Logging Level in ParquetOutputFormat (apache#745) PARQUET-2046: Upgrade Apache POM to 23 (apache#904) PARQUET-2048: Deprecate BaseRecordReader (apache#906) PARQUET-1922: Deprecate IOExceptionUtils (apache#825) PARQUET-2037: Write INT96 with parquet-avro (apache#901) PARQUET-2044: Enable ZSTD buffer pool by default (apache#903) PARQUET-2038: Upgrade Jackson version used in parquet encryption. (apache#898) Revert "[WIP] Refactor GroupReadSupport to unuse deprecated api (apache#894)" PARQUET-2027: Fix calculating directory offset for merge (apache#896) [WIP] Refactor GroupReadSupport to unuse deprecated api (apache#894) PARQUET-2030: Expose page size row check configurations to ParquetWriter.Builder (apache#895) PARQUET-2031: Upgrade to parquet-format 2.9.0 (apache#897) PARQUET-1448: Review of ParquetFileReader (apache#892) PARQUET-2020: Remove deprecated modules (apache#888) PARQUET-2025: Update Snappy version to 1.1.8.3 (apache#893) PARQUET-2022: ZstdDecompressorStream should close `zstdInputStream` (apache#889) PARQUET-1982: Random access to row groups in ParquetFileReader (apache#871) ... # Conflicts: # parquet-column/src/main/java/org/apache/parquet/example/data/simple/SimpleGroup.java # parquet-hadoop/pom.xml # parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java # parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java
ZstdDecompressorStreamshould close its resource becauseCompressionInputStream.closecloses only the inner stream.Make sure you have checked all steps below.
Jira
Tests
Use revised test case.
Commits
Documentation