Updated gradle and java version to 24 #1436

owaiskazi19 · 2025-07-10T19:29:15Z

Description

Updated gradle and java version to 24
Used opensearch CI for bwc tests
Updated checkout and java version for bwc tests
Updated BWC version to 3.2

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

peterzhuamazon · 2025-07-10T19:33:19Z

Seems like there is compiling error here @owaiskazi19 ?

Thanks.

github checks failure.

owaiskazi19 · 2025-07-10T19:43:37Z

Seems like there is compiling error here @owaiskazi19 ?

Weird, working fine on local

BUILD SUCCESSFUL in 21s

@martin-gaievski @bzhangam can rerun the checks here?

heemin32 · 2025-07-10T20:01:30Z

Rerun will fail I believe.

/__w/neural-search/neural-search/src/main/java/org/opensearch/neuralsearch/util/SemanticMappingUtils.java:399: error: cannot find symbol
        @NonNull final NeuralQueryTargetFieldConfig.NeuralQueryTargetFieldConfigBuilder targetFieldConfigBuilder
                                                   ^
  symbol:   class NeuralQueryTargetFieldConfigBuilder
  location: class NeuralQueryTargetFieldConfig
/__w/neural-search/neural-search/src/main/java/org/opensearch/neuralsearch/executors/HybridQueryExecutorCollector.java:27: error: cannot find symbol
    @Getter(onMethod_ = { @Synchronized })
            ^
  symbol:   method onMethod_()
  location: @interface Getter

.github/workflows/CI.yml

bzhangam · 2025-07-10T21:25:27Z

Rerun will fail I believe.

/__w/neural-search/neural-search/src/main/java/org/opensearch/neuralsearch/util/SemanticMappingUtils.java:399: error: cannot find symbol
        @NonNull final NeuralQueryTargetFieldConfig.NeuralQueryTargetFieldConfigBuilder targetFieldConfigBuilder
                                                   ^
  symbol:   class NeuralQueryTargetFieldConfigBuilder
  location: class NeuralQueryTargetFieldConfig
/__w/neural-search/neural-search/src/main/java/org/opensearch/neuralsearch/executors/HybridQueryExecutorCollector.java:27: error: cannot find symbol
    @Getter(onMethod_ = { @Synchronized })
            ^
  symbol:   method onMethod_()
  location: @interface Getter

I think the issue is related to the lombok and somehow it cannot generate the codes properly in git actions. Probably we can try to use a newer version https://plugins.gradle.org/plugin/io.freefair.lombok?

codecov · 2025-07-10T21:55:47Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (8c5fb57) to head (cf13285).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #1436       +/-   ##
============================================
- Coverage     79.95%       0   -79.96%     
============================================
  Files           160       0      -160     
  Lines          8584       0     -8584     
  Branches       1396       0     -1396     
============================================
- Hits           6863       0     -6863     
+ Misses         1186       0     -1186     
+ Partials        535       0      -535

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

owaiskazi19 · 2025-07-10T21:58:58Z

BWC tests fails on the knn side

»  fatal error in thread [main], exiting
»  java.util.ServiceConfigurationError: org.apache.lucene.codecs.KnnVectorsFormat: Provider org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsFormat could not be instantiated
»  	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:552)
»  	at ava.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:712)
»  	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:672)
»  	at java.base/java.util.ServiceLoader$2.next(ServiceLoader.java:1256)
»  	at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:68)
»  	at org.apache.lucene.codecs.KnnVectorsFormat.reloadKnnVectorsFormat(KnnVectorsFormat.java:82)
»  	at org.opensearch.plugins.PluginsService.reloadLuceneSPI(PluginsService.java:842)
»  	at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:795)
»  	at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:615)
»  	at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:229)
»  	at org.opensearch.node.Node.<init>(Node.java:536)
»  	at org.opensearch.node.Node.<init>(Node.java:464)
»  	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:249)
»  	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:249)
»  	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:411)
»  	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:168)
»  	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:159)
»  	at org.opensearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:110)
»  	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
»  	at org.opensearch.cli.Command.main(Command.java:101)
»  	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:125)
»  	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:91)
»  Caused by: java.lang.ExceptionInInitializerError
»  	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsFormat.<init>(NativeEngines990KnnVectorsFormat.java:50)
»  	at org.opensearch.knn.index.codec.KNN990Codec.NativeEngines990KnnVectorsFormat.<init>(NativeEngines990KnnVectorsFormat.java:42)
»  	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
»  	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
»  	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
»  	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:707)
»  	... 20 more

cc: @naveentatikonda @navneet1v

heemin32 · 2025-07-14T22:47:43Z

Pending opensearch-project/k-NN#2792

owaiskazi19 · 2025-07-16T17:21:11Z

Pending opensearch-project/k-NN#2792

opensearch-project/k-NN#2792 is merged. Need to rerun the checks

owaiskazi19 · 2025-07-16T17:42:17Z

Failing with

Caused by: java.lang.SecurityException: Denied OPEN (read) access to file: /sys/fs/cgroup/system.slice/hosted-compute-agent.service/memory.max, domain: ProtectionDomain  (file:/home/runner/work/neural-search/neural-search/qa/rolling-upgrade/build/testclusters/neuralSearchBwcCluster-rolling-0/distro/3.2.0-ARCHIVE/plugins/opensearch-knn/opensearch-knn-3.2.0.0-SNAPSHOT.jar <no signer certificates>)
»   java.net.URLClassLoader@2e4eda17
»   <no principals>
»   java.security.Permissions@3a2546d6 (
»  )
»

we will need to add permission java.io.FilePermission "/sys/fs/cgroup/system.slice/hosted-compute-agent.service/memory.max", "read"; permission.

owaiskazi19 · 2025-07-17T17:57:52Z

BWC is still failing with the below permission even when it's added on core side: https://github.com/opensearch-project/OpenSearch/pull/18771/files

 »  Caused by: java.lang.SecurityException: Denied OPEN (read) access to file: /sys/fs/cgroup/system.slice/hosted-compute-agent.service/memory.max, domain: ProtectionDomain  (file:/home/runner/work/neural-search/neural-search/qa/rolling-upgrade/build/testclusters/neuralSearchBwcCluster-rolling-0/distro/3.2.0-ARCHIVE/lib/opensearch-3.2.0-SNAPSHOT.jar <no signer certificates>)
»   jdk.internal.loader.ClassLoaders$AppClassLoader@659e0bfd
»   <no principals>
»   java.security.Permissions@4df7d9ee (
»  )
»  
»

I also tried to add the permission to the plugin explicitly.
Should we add this permission on knn side as well?

 Caused by: java.lang.SecurityException: Denied OPEN (read) access to file: /sys/fs/cgroup/system.slice/runner-provisioner.service/memory.max, domain: ProtectionDomain  (file:/home/runner/work/neural-search/neural-search/qa/rolling-upgrade/build/testclusters/neuralSearchBwcCluster-rolling-0/distro/3.2.0-ARCHIVE/plugins/opensearch-knn/opensearch-knn-3.2.0.0-SNAPSHOT.jar <no signer certificates>)

cc: @reta @cwperks

cwperks · 2025-07-17T19:38:02Z

@owaiskazi19 Why is BWC testing 2.19 to 3.1? Shouldn't it be on 3.2 now? Make sure #1406 is included in your feature branch.

owaiskazi19 · 2025-07-17T19:42:02Z

@owaiskazi19 Why is BWC testing 2.19 to 3.1? Shouldn't it be on 3.2 now? Make sure #1406 is included in your feature branch.

I rebased and actually updated bwc version in this PR. But now pulled in this PR to run with 3.2

Signed-off-by: Owais <[email protected]>

.github/workflows/backwards_compatibility_tests_workflow.yml

Signed-off-by: Owais <[email protected]>

ryanbogan

LGTM

heemin32 · 2025-07-18T00:07:50Z

Thanks for the change. The failure in BWC is from a bug in the test itself.

owaiskazi19 requested review from VijayanB, bzhangam, heemin32, jmazanec15, junqiu-lei, martin-gaievski, minalsha, model-collapse, naveentatikonda, navneet1v, sean-zheng-amazon, vamshin, vibrantvarun, yuye-aws, zane-neo and zhichao-aws as code owners July 10, 2025 19:29

owaiskazi19 added the skip-changelog label Jul 10, 2025

peterzhuamazon previously approved these changes Jul 10, 2025

View reviewed changes

peterzhuamazon self-requested a review July 10, 2025 19:33

martin-gaievski reviewed Jul 10, 2025

View reviewed changes

.github/workflows/CI.yml Show resolved Hide resolved

owaiskazi19 closed this Jul 17, 2025

owaiskazi19 reopened this Jul 17, 2025

owaiskazi19 force-pushed the update-gradle branch from c1a189e to 7306d6b Compare July 17, 2025 19:40

owaiskazi19 added 13 commits July 17, 2025 15:06

Updated gradle and java version to 24

f9c1332

Signed-off-by: Owais <[email protected]>

Updated lombok version

6ebcf33

Signed-off-by: Owais <[email protected]>

Added accessUnixDomainSocket to address security manager deprecation

55d4e26

Signed-off-by: Owais <[email protected]>

Removed the permission

75e40a8

Signed-off-by: Owais <[email protected]>

Added permission for memory.max

118715f

Signed-off-by: Owais <[email protected]>

Removed plugin specific permission

4d39332

Signed-off-by: Owais <[email protected]>

Updated bwc version

9b91054

Signed-off-by: Owais <[email protected]>

Updated os version

1685235

Signed-off-by: Owais <[email protected]>

Added read permission

9682fac

Signed-off-by: Owais <[email protected]>

Continue the tests on failure

7328529

Signed-off-by: Owais <[email protected]>

Use OS CI image

567c720

Signed-off-by: Owais <[email protected]>

Incremented setup java version

e3aacbd

Signed-off-by: Owais <[email protected]>

Fixed CI

77ecea3

Signed-off-by: Owais <[email protected]>

owaiskazi19 force-pushed the update-gradle branch from e3eaf6f to 77ecea3 Compare July 17, 2025 22:22

peterzhuamazon reviewed Jul 17, 2025

View reviewed changes

.github/workflows/backwards_compatibility_tests_workflow.yml Outdated Show resolved Hide resolved

owaiskazi19 added 2 commits July 17, 2025 15:32

Added root permission

734a1ab

Signed-off-by: Owais <[email protected]>

Removed the permission

cf13285

Signed-off-by: Owais <[email protected]>

owaiskazi19 requested review from martin-gaievski and peterzhuamazon July 17, 2025 23:06

ryanbogan approved these changes Jul 17, 2025

View reviewed changes

heemin32 approved these changes Jul 18, 2025

View reviewed changes

martin-gaievski approved these changes Jul 18, 2025

View reviewed changes

martin-gaievski merged commit d0f83a4 into opensearch-project:main Jul 18, 2025
55 of 61 checks passed

Updated gradle and java version to 24 #1436

Updated gradle and java version to 24 #1436

Uh oh!

Conversation

owaiskazi19 commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

peterzhuamazon commented Jul 10, 2025

Uh oh!

owaiskazi19 commented Jul 10, 2025

Uh oh!

heemin32 commented Jul 10, 2025

Uh oh!

Uh oh!

bzhangam commented Jul 10, 2025

Uh oh!

codecov bot commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

owaiskazi19 commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

heemin32 commented Jul 14, 2025

Uh oh!

owaiskazi19 commented Jul 16, 2025

Uh oh!

owaiskazi19 commented Jul 16, 2025

Uh oh!

owaiskazi19 commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cwperks commented Jul 17, 2025

Uh oh!

owaiskazi19 commented Jul 17, 2025

Uh oh!

Uh oh!

ryanbogan left a comment

Choose a reason for hiding this comment

Uh oh!

heemin32 commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

owaiskazi19 commented Jul 10, 2025 •

edited

Loading

codecov bot commented Jul 10, 2025 •

edited

Loading

owaiskazi19 commented Jul 10, 2025 •

edited

Loading

owaiskazi19 commented Jul 17, 2025 •

edited

Loading

heemin32 commented Jul 18, 2025 •

edited

Loading