update sandbox check to publish latest UnifiedQueryAPI jars to maven local - #21569
Merged
Conversation
…local. This is a temporary work-around to publish latest unified jars built to integrate with analytics-engine to maven local - we are working off of a feature branch of sql in the short term. if the step fails, the build will continue with jars from maven central. This will be removed once changes for analytics-engine in sql are merged to mainline. Signed-off-by: Marc Handalian <marc.handalian@gmail.com>
Contributor
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit 142763d.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
mch2
added
skip-diff-analyzer
andrross
reviewed
May 8, 2026
andrross
approved these changes
May 8, 2026
RyanL1997
added a commit
to RyanL1997/OpenSearch
that referenced
this pull request
May 9, 2026
The remote OpenSearch Snapshots maven repo (ci.opensearch.org/ci/dbc/snapshots) only republishes from sql/main, not from sql/feature/mustang-ppl-integration, so its 3.7.0.0-SNAPSHOT jars trail the feature branch by however many merges (currently missing PPL_REX_MAX_MATCH_LIMIT, CALCITE_ENGINE_ENABLED, …). The sandbox-check workflow's pre-step opensearch-project#21569 publishes feature-branch unified-query jars to mavenLocal, but Gradle's default SNAPSHOT resolution weighs the remote's explicit <buildNumber>/<timestamp> metadata higher than mavenLocal's <localCopy>true>, so the stale remote wins even when mavenLocal has a newer <lastUpdated>. Confirmed via dependencyInsight: every consumer was binding unified-query-api:3.7.0.0-SNAPSHOT:20260507.224009-12 (60kB, 42 classes, no PPL_REX_MAX_MATCH_LIMIT field reference) instead of the locally-published 3.7.0.0-SNAPSHOT (29kB, 21 classes, has the field). The runtime cluster inherited that stale class via the test-ppl-frontend plugin bundle, which is why every IT touching `rex` failed plan-time with `NullPointerException: Cannot invoke "java.lang.Integer.intValue()" because the return value of "Settings.getSettingValue(PPL_REX_MAX_MATCH_LIMIT)" is null` once the unified path tried to read the setting. Fix: tell the OpenSearch Snapshots remote to refuse `org.opensearch.query` artifacts via mavenContent { excludeGroup }. Three sites declare the remote: * sandbox/build.gradle subprojects { repositories } — applies to every sandbox subproject including qa. * sandbox/plugins/analytics-backend-datafusion/build.gradle — own declaration; left in place for module isolation, filtered identically. * sandbox/plugins/test-ppl-frontend/build.gradle — also pin mavenLocal as the only source for org.opensearch.query so the bundlePlugin task bundles the freshly-published feature-branch jar rather than the stale timestamped one Gradle would otherwise pick. Verified locally: bundled unified-query-api drops 60kB → 29kB, the UnifiedQueryContext$Builder constant pool now references PPL_REX_MAX_MATCH_LIMIT, and RexCommandIT goes 0/16 → 16/16 against the same locally-published jars the CI workflow already produces. Drop this filter once the SQL feature branch merges to sql/main and the remote OpenSearch Snapshots repo catches up — at that point every 3.7.0.0-SNAPSHOT publish will carry the rex max-match default and the mavenLocal preference becomes redundant. Signed-off-by: Jialiang Liang <jiallian@amazon.com>
RyanL1997
added a commit
to RyanL1997/OpenSearch
that referenced
this pull request
May 9, 2026
The remote OpenSearch Snapshots maven repo (ci.opensearch.org/ci/dbc/snapshots) only republishes from sql/main, not from sql/feature/mustang-ppl-integration, so its 3.7.0.0-SNAPSHOT jars trail the feature branch by however many merges (currently missing PPL_REX_MAX_MATCH_LIMIT, CALCITE_ENGINE_ENABLED, …). The sandbox-check workflow's pre-step opensearch-project#21569 publishes feature-branch unified-query jars to mavenLocal, but Gradle's default SNAPSHOT resolution weighs the remote's explicit <buildNumber>/<timestamp> metadata higher than mavenLocal's <localCopy>true>, so the stale remote wins even when mavenLocal has a newer <lastUpdated>. Confirmed via dependencyInsight: every consumer was binding unified-query-api:3.7.0.0-SNAPSHOT:20260507.224009-12 (60kB, 42 classes, no PPL_REX_MAX_MATCH_LIMIT field reference) instead of the locally-published 3.7.0.0-SNAPSHOT (29kB, 21 classes, has the field). The runtime cluster inherited that stale class via the test-ppl-frontend plugin bundle, which is why every IT touching `rex` failed plan-time with `NullPointerException: Cannot invoke "java.lang.Integer.intValue()" because the return value of "Settings.getSettingValue(PPL_REX_MAX_MATCH_LIMIT)" is null` once the unified path tried to read the setting. Fix: tell the OpenSearch Snapshots remote to refuse `org.opensearch.query` artifacts via mavenContent { excludeGroup }. Three sites declare the remote: * sandbox/build.gradle subprojects { repositories } — applies to every sandbox subproject including qa. * sandbox/plugins/analytics-backend-datafusion/build.gradle — own declaration; left in place for module isolation, filtered identically. * sandbox/plugins/test-ppl-frontend/build.gradle — also pin mavenLocal as the only source for org.opensearch.query so the bundlePlugin task bundles the freshly-published feature-branch jar rather than the stale timestamped one Gradle would otherwise pick. Verified locally: bundled unified-query-api drops 60kB → 29kB, the UnifiedQueryContext$Builder constant pool now references PPL_REX_MAX_MATCH_LIMIT, and RexCommandIT goes 0/16 → 16/16 against the same locally-published jars the CI workflow already produces. Drop this filter once the SQL feature branch merges to sql/main and the remote OpenSearch Snapshots repo catches up — at that point every 3.7.0.0-SNAPSHOT publish will carry the rex max-match default and the mavenLocal preference becomes redundant. Signed-off-by: Jialiang Liang <jiallian@amazon.com>
mch2
pushed a commit
that referenced
this pull request
May 9, 2026
…napshots to mavenLocal (#21578) * Pin org.opensearch.query:* (unified-query-*) artifacts to mavenLocal The remote OpenSearch Snapshots maven repo (ci.opensearch.org/ci/dbc/snapshots) only republishes from sql/main, not from sql/feature/mustang-ppl-integration, so its 3.7.0.0-SNAPSHOT jars trail the feature branch by however many merges (currently missing PPL_REX_MAX_MATCH_LIMIT, CALCITE_ENGINE_ENABLED, …). The sandbox-check workflow's pre-step #21569 publishes feature-branch unified-query jars to mavenLocal, but Gradle's default SNAPSHOT resolution weighs the remote's explicit <buildNumber>/<timestamp> metadata higher than mavenLocal's <localCopy>true>, so the stale remote wins even when mavenLocal has a newer <lastUpdated>. Confirmed via dependencyInsight: every consumer was binding unified-query-api:3.7.0.0-SNAPSHOT:20260507.224009-12 (60kB, 42 classes, no PPL_REX_MAX_MATCH_LIMIT field reference) instead of the locally-published 3.7.0.0-SNAPSHOT (29kB, 21 classes, has the field). The runtime cluster inherited that stale class via the test-ppl-frontend plugin bundle, which is why every IT touching `rex` failed plan-time with `NullPointerException: Cannot invoke "java.lang.Integer.intValue()" because the return value of "Settings.getSettingValue(PPL_REX_MAX_MATCH_LIMIT)" is null` once the unified path tried to read the setting. Fix: tell the OpenSearch Snapshots remote to refuse `org.opensearch.query` artifacts via mavenContent { excludeGroup }. Three sites declare the remote: * sandbox/build.gradle subprojects { repositories } — applies to every sandbox subproject including qa. * sandbox/plugins/analytics-backend-datafusion/build.gradle — own declaration; left in place for module isolation, filtered identically. * sandbox/plugins/test-ppl-frontend/build.gradle — also pin mavenLocal as the only source for org.opensearch.query so the bundlePlugin task bundles the freshly-published feature-branch jar rather than the stale timestamped one Gradle would otherwise pick. Verified locally: bundled unified-query-api drops 60kB → 29kB, the UnifiedQueryContext$Builder constant pool now references PPL_REX_MAX_MATCH_LIMIT, and RexCommandIT goes 0/16 → 16/16 against the same locally-published jars the CI workflow already produces. Drop this filter once the SQL feature branch merges to sql/main and the remote OpenSearch Snapshots repo catches up — at that point every 3.7.0.0-SNAPSHOT publish will carry the rex max-match default and the mavenLocal preference becomes redundant. Signed-off-by: Jialiang Liang <jiallian@amazon.com> * Wire mavenLocal into sandbox subprojects + bump analytics-engine to 3.7 CI fallout from the prior commit's `excludeGroup 'org.opensearch.query'` filter on the OpenSearch Snapshots remote: the parent subprojects block no longer carried mavenLocal, so analytics-engine's testImplementation / internalClusterTest configurations had no repository at all serving org.opensearch.query, failing with `Could not find org.opensearch.query:unified-query-api:3.6.0.0-SNAPSHOT` (and -core / -ppl). Two pieces: 1. sandbox/build.gradle subprojects { repositories } — also declare mavenLocal scoped to the org.opensearch.query group via mavenContent { includeGroup }. mavenLocal becomes the authoritative source for unified-query SNAPSHOTs (populated by the sandbox-check workflow's publishUnifiedQueryPublicationToMavenLocal pre-step) without leaking into resolution for any other group. 2. sandbox/plugins/analytics-engine/build.gradle — bump sqlUnifiedQueryVersion from 3.6.0.0-SNAPSHOT → 3.7.0.0-SNAPSHOT. The 3.6 jars don't exist in mavenLocal (only the 3.7 feature-branch build does), so the older pin was the proximate cause of the CI resolution failure. Aligning with test-ppl-frontend's already-3.7 declaration also keeps the unified-query consumers consistent. Signed-off-by: Jialiang Liang <jiallian@amazon.com> --------- Signed-off-by: Jialiang Liang <jiallian@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a temporary work-around to publish latest unified jars built to integrate with analytics-engine to maven local - we are working off of a feature branch of sql in the short term. if the step fails, the build will continue with jars from maven central. This will be removed once changes for analytics-engine in sql are merged to mainline.
Description
[Describe what this change achieves]
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.