Skip to content

Conversation

@jackye1995
Copy link
Contributor

When used in BaseMetastoreTableOperations, in TableMetadataParser.internalWrite(), an output stream is closed twice because of the closure of the output stream and its writer:

try (OutputStream ou = isGzip ? new GZIPOutputStream(stream) : stream;
         OutputStreamWriter writer = new OutputStreamWriter(ou, StandardCharsets.UTF_8)) {
   ...
}

This causes issue in S3OutputStream because the stream uploads the file from local to S3 during the call to close.

Also, S3FileIO cannot be properly initialized when there are multiple HTTP client builders available in the Java class path at runtime, so AwsClientUtil is used to provide a stable default client for S3FileIO to use.

Copy link
Contributor

@rdblue rdblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I'd like to make sure @danielcweeks agrees.

@danielcweeks
Copy link
Contributor

+1 LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants