Skip to content

AWS: Refactor S3FileIOProperties to use common builder interface #13183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RussellSpitzer merged 2 commits into from
Jun 30, 2025
Merged

AWS: Refactor S3FileIOProperties to use common builder interface #13183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c70375c
AWS: Refactor S3FileIOProperties to use common builder interface
devinrsmith May 29, 2025
2a163b6
fix javadoc
devinrsmith May 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions aws/src/main/java/org/apache/iceberg/aws/AwsClientFactories.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@
import software.amazon.awssdk.services.glue.GlueClientBuilder;
import software.amazon.awssdk.services.kms.KmsClient;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.S3BaseClientBuilder;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3ClientBuilder;
import software.amazon.awssdk.services.s3.S3Configuration;

public class AwsClientFactories {
Expand Down Expand Up @@ -204,7 +204,7 @@ public static SdkHttpClient.Builder configureHttpClientBuilder(String httpClient
* Configure the endpoint setting for a client
*
* @deprecated Not for public use. To configure the endpoint for a client, please use {@link
* S3FileIOProperties#applyEndpointConfigurations(S3ClientBuilder)}, {@link
* S3FileIOProperties#applyEndpointConfigurations(S3BaseClientBuilder)}, {@link
* AwsProperties#applyGlueEndpointConfigurations(GlueClientBuilder)}, or {@link
* AwsProperties#applyDynamoDbEndpointConfigurations(DynamoDbClientBuilder)} accordingly. It
* will be removed in 2.0.0
Expand Down Expand Up @@ -235,8 +235,8 @@ public static S3Configuration s3Configuration(
* Build an AwsBasicCredential object
*
* @deprecated Not for public use. To configure the credentials for a s3 client, please use {@link
* S3FileIOProperties#applyCredentialConfigurations(AwsClientProperties, S3ClientBuilder)} in
* AwsProperties. It will be removed in 2.0.0.
* S3FileIOProperties#applyCredentialConfigurations(AwsClientProperties, S3BaseClientBuilder)}
* in AwsProperties. It will be removed in 2.0.0.
*/
@Deprecated
static AwsCredentialsProvider credentialsProvider(
Expand Down
19 changes: 2 additions & 17 deletions aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@
import software.amazon.awssdk.core.retry.conditions.RetryCondition;
import software.amazon.awssdk.core.retry.conditions.RetryOnExceptionsCondition;
import software.amazon.awssdk.core.retry.conditions.TokenBucketRetryCondition;
import software.amazon.awssdk.services.s3.S3AsyncClientBuilder;
import software.amazon.awssdk.services.s3.S3BaseClientBuilder;
import software.amazon.awssdk.services.s3.S3ClientBuilder;
import software.amazon.awssdk.services.s3.S3Configuration;
Expand Down Expand Up @@ -1022,30 +1021,16 @@ public <T extends S3ClientBuilder> void applySignerConfiguration(T builder) {
}

/**
* Override the endpoint for an S3 client.
* Override the endpoint for an S3 sync or async client.
*
* <p>Sample usage:
*
* <pre>
* S3Client.builder().applyMutation(s3FileIOProperties::applyEndpointConfigurations)
* </pre>
*/
public <T extends S3ClientBuilder> void applyEndpointConfigurations(T builder) {
if (endpoint != null) {
builder.endpointOverride(URI.create(endpoint));
}
}

/**
* Override the endpoint for an S3 async client
*
* <p>Sample usage:
*
* <pre>
* S3AsyncClient.builder().applyMutation(s3FileIOProperties::applyEndpointConfigurations)
* </pre>
*/
public <T extends S3AsyncClientBuilder> void applyEndpointConfigurations(T builder) {
public <T extends S3BaseClientBuilder<T, ?>> void applyEndpointConfigurations(T builder) {
if (endpoint != null) {
builder.endpointOverride(URI.create(endpoint));
}
Expand Down