Skip to content

Add aws-java-sdk-core as runtime dependency for Redshift JDBC driver#25265

Merged
imjalpreet merged 1 commit intoprestodb:masterfrom
adkharat:redshift-jdbc-aws-dependency
Jun 17, 2025
Merged

Add aws-java-sdk-core as runtime dependency for Redshift JDBC driver#25265
imjalpreet merged 1 commit intoprestodb:masterfrom
adkharat:redshift-jdbc-aws-dependency

Conversation

@adkharat
Copy link
Contributor

@adkharat adkharat commented Jun 9, 2025

Description

Git Issue: #25247

Amazon Redshift JDBC driver version 2.1.0.32 internally depends on com.amazonaws.util.StringUtils class from the AWS SDK v1.

This com.amazonaws.util.StringUtils class is not bundled inside the Redshift JDBC driver jar, and it must be available on the classpath.

Added aws-java-sdk-core as a runtime dependency to resolve this issue.

Motivation and Context

The Presto Redshift connector fails with a NoClassDefFoundError during table operations like CREATE TABLE because it tries to invoke a metadata function that uses this class.

Impact

Test Plan

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Redshift Connector Changes
* Fix Redshift connector runtime failure due to missing dependency on ``com.amazonaws.util.StringUtils``.
Add ``aws-java-sdk-core`` as a runtime dependency to support Redshift JDBC driver (v2.1.0.32) which relies on this class for metadata operations.

@adkharat adkharat requested a review from a team as a code owner June 9, 2025 12:30
@adkharat adkharat requested a review from jaystarshot June 9, 2025 12:30
@prestodb-ci prestodb-ci added the from:IBM PR from IBM label Jun 9, 2025
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jun 9, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: adkharat / name: Ajay Kharat (b837afc)

@prestodb-ci prestodb-ci requested review from a team, imsayari404 and nmahadevuni and removed request for a team June 9, 2025 12:30
@adkharat adkharat force-pushed the redshift-jdbc-aws-dependency branch from a549ce6 to 4d667bd Compare June 9, 2025 12:34
@adkharat
Copy link
Contributor Author

adkharat commented Jun 9, 2025

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-core</artifactId>
    <scope>runtime</scope>
</dependency>

Redshift JDBC driver internally use com.amazonaws.util.StringUtils at runtime. So added scope of runtime.

Copy link
Member

@imjalpreet imjalpreet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment, otherwise LGTM.

If possible, could you attach a screenshot of a successful query to Redshift after adding this fix?

Comment on lines +114 to +124
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.12.782</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this, aws-java-sdk-core is already included in the root POM.

presto/pom.xml

Lines 1760 to 1778 in 91a8685

<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>${dep.aws-sdk.version}</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</exclusion>
</exclusions>
</dependency>

@steveburnett
Copy link
Contributor

Thanks for the release note! Just a nit. Should this entry be in Redshift Connector Changes instead of General Changes, like so?

== RELEASE NOTES ==

Redshift Connector Changes
* Fix Redshift connector runtime failure due to missing dependency on ``com.amazonaws.util.StringUtils``.
Add ``aws-java-sdk-core`` as a runtime dependency to support Redshift JDBC driver (v2.1.0.32) which relies on this class for metadata operations.

@adkharat
Copy link
Contributor Author

adkharat commented Jun 11, 2025

@imjalpreet Correct, aws-java-sdk-core is already included in root pom.xml

  • Scenario-1: Removing aws-java-sdk-core from presto-redshift/pom.xml causes a runtime error:
    Query failed: com/amazonaws/util/StringUtils

I think, Root pom.xml defines it, but not for plugin packaging:
The presto-redshift module produces a plugin (<packaging>presto-plugin</packaging>), which requires all needed runtime dependencies to be included explicitly in its own module, even if they exist in the parent pom.xml.

Screenshot 2025-06-11 at 12 27 48 PM
  • Scenario-2: On keeping aws-java-sdk-core dependency in presto-redshift/pom.xml, I see CREATE TABLE execute successfully.
Screenshot 2025-06-11 at 12 09 21 PM

@adkharat
Copy link
Contributor Author

Thanks for the release note! Just a nit. Should this entry be in Redshift Connector Changes instead of General Changes, like so?

== RELEASE NOTES ==

Redshift Connector Changes
* Fix Redshift connector runtime failure due to missing dependency on ``com.amazonaws.util.StringUtils``.
Add ``aws-java-sdk-core`` as a runtime dependency to support Redshift JDBC driver (v2.1.0.32) which relies on this class for metadata operations.

Correct. Updated

@adkharat adkharat requested a review from imjalpreet June 11, 2025 12:44
Copy link
Member

@imjalpreet imjalpreet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scenario-1: Removing aws-java-sdk-core from presto-redshift/pom.xml causes a runtime error:
Query failed: com/amazonaws/util/StringUtils

I think, Root pom.xml defines it, but not for plugin packaging:
The presto-redshift module produces a plugin (presto-plugin), which requires all needed runtime dependencies to be included explicitly in its own module, even if they exist in the parent pom.xml.

We need to keep the dependency in presto-redshift pom.xml, my comment was about removing the DependencyManagement section, as we don't need it.

removed dep manag from redshift pom
@adkharat adkharat force-pushed the redshift-jdbc-aws-dependency branch from c561715 to b837afc Compare June 12, 2025 03:20
@adkharat
Copy link
Contributor Author

Scenario-1: Removing aws-java-sdk-core from presto-redshift/pom.xml causes a runtime error:
Query failed: com/amazonaws/util/StringUtils

I think, Root pom.xml defines it, but not for plugin packaging:
The presto-redshift module produces a plugin (presto-plugin), which requires all needed runtime dependencies to be included explicitly in its own module, even if they exist in the parent pom.xml.

We need to keep the dependency in presto-redshift pom.xml, my comment was about removing the DependencyManagement section, as we don't need it.

@imjalpreet Done

Copy link
Member

@imjalpreet imjalpreet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@prestodb-ci
Copy link
Contributor

@ethanyzhang imported this issue as lakehouse/presto #25265

@imjalpreet imjalpreet merged commit 456584f into prestodb:master Jun 17, 2025
217 of 219 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants