Force core :server error prone annotations version #2885

finnegancarroll · 2025-09-18T19:36:17Z

Description

There seems to be a newly introduced dependency conflict with error_prone_annotations.
opensearch-project/OpenSearch@b9c5bc7#diff-56915d53ea588a229a43cca18bc96695cb3aec2812cb24f8514062ca0d9ccb41R101-R103

2.26.1 is included with guava:33.2.1-jre so KNN sees both versions.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileTestFixturesJava'.
> Could not resolve all files for configuration ':testFixturesCompileClasspath'.
   > Could not resolve com.google.errorprone:error_prone_annotations:2.26.1.
     Required by:
         root project : > com.google.guava:guava:33.2.1-jre
      > Conflict found for module 'com.google.errorprone:error_prone_annotations': between versions 2.41.0 and 2.26.1
> There is 1 more failure with an identical cause.

This new version of error_prone_annotations is visible through :test:framework.

./gradlew :dependencyInsight --configuration testFixturesCompileClasspath --dependency com.google.errorprone:error_prone_annotations


> Task :dependencyInsight
com.google.errorprone:error_prone_annotations:2.26.1 FAILED
   Failures:
      - Could not resolve com.google.errorprone:error_prone_annotations:2.26.1.
          - Conflict found for module 'com.google.errorprone:error_prone_annotations': between versions 2.41.0 and 2.26.1

com.google.errorprone:error_prone_annotations:2.26.1 FAILED
\--- com.google.guava:guava:33.2.1-jre
     \--- root project :
          \--- root project : (*)

com.google.errorprone:error_prone_annotations:2.41.0 FAILED
   Failures:
      - Could not resolve com.google.errorprone:error_prone_annotations:2.41.0. (already reported)

com.google.errorprone:error_prone_annotations:2.41.0 FAILED
\--- org.opensearch:opensearch:3.3.0-SNAPSHOT:20250917.232945-101
     +--- root project :
     |    \--- root project : (*)
     \--- org.opensearch.test:framework:3.3.0-SNAPSHOT:20250917.232945-101
          \--- root project : (*)

which in turn gets it from taking a dependency on :server:

./gradlew :test:framework:dependencyInsight --dependency com.google.errorprone:error_prone_annotations --configuration compileClasspath

com.google.errorprone:error_prone_annotations:2.41.0
\--- project :server
     \--- compileClasspath

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

finnegancarroll · 2025-09-18T21:38:35Z

Even when moving the knn failureaccess to implementation it is still not available at runtime it seems...

2025-09-18T20:05:52.2431340Z »  fatal error in thread [opensearch[integTest-0][search][T#1]], exiting
2025-09-18T20:05:52.2432107Z »  java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
2025-09-18T20:05:52.2432711Z »  	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
2025-09-18T20:05:52.2433179Z »  	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:962)
2025-09-18T20:05:52.2433971Z »  	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:132)
2025-09-18T20:05:52.2434457Z »  	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:426)
2025-09-18T20:05:52.2434913Z »  	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:339)
2025-09-18T20:05:52.2435321Z »  	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:557)
2025-09-18T20:05:52.2435726Z »  	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:490)

navneet1v · 2025-09-18T22:09:33Z

BTW I also have my attempt of the same: #2878

finnegancarroll · 2025-09-18T23:49:44Z

With only forcing guava resolution failure access does not appear in the distribution bundle as it is now no longer included through grpc transport module due to move to SPI.

unzip -l build/distributions/opensearch-knn-*.zip

Archive:  build/distributions/opensearch-knn-3.3.0.0-SNAPSHOT.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
     1929  02-01-1980 00:00   plugin-descriptor.properties
      787  02-01-1980 00:00   plugin-security.policy
  1071363  02-01-1980 00:00   opensearch-knn-3.3.0.0-SNAPSHOT.jar
    32203  02-01-1980 00:00   remote-index-build-client-3.3.0.0-SNAPSHOT.jar
   702952  02-01-1980 00:00   commons-lang3-3.18.0.jar
  1375665  02-01-1980 00:00   jna-platform-5.16.0.jar
   979563  02-01-1980 00:00   oshi-core-6.4.13.jar
    69908  02-01-1980 00:00   slf4j-api-2.0.17.jar
   284220  02-01-1980 00:00   commons-lang-2.6.jar
   911620  02-01-1980 00:00   httpclient5-5.4.4.jar
   908601  02-01-1980 00:00   httpcore5-5.3.4.jar
   241692  02-01-1980 00:00   httpcore5-h2-5.3.4.jar
       51  02-01-1980 00:00   NOTICE.txt
    20535  02-01-1980 00:00   LICENSE.txt
---------                     -------
  6601089                     14 files

Including failure access ass an explicit dependency for test implementation gets failure access into the bundle:

    implementation 'com.google.guava:failureaccess:1.0.2'
    testImplementation 'com.google.guava:failureaccess:1.0.2'
    testFixturesImplementation 'com.google.guava:failureaccess:1.0.2'

->

 unzip -l build/distributions/opensearch-knn-*.zip | grep failure

     4740  02-01-1980 00:00   failureaccess-1.0.2.jar

But still the class loader cannot find this definition...
Confusing why this is the case. The usage here comes from KNN as well in org.opensearch.knn.index.memory.NativeMemoryCacheManager.get(NativeMemoryCacheManager.java:441).

=== Standard error of node `node{::integTest-0}` ===
»   ↓ last 40 non error or warning messages from /Users/carrofin/fdev/repos/k-NN/build/testclusters/integTest-0/logs/opensearch.stderr.log ↓
» WARNING: Using incubator modules: jdk.incubator.vector
»  WARNING: Unknown module: org.apache.arrow.memory.core specified to --add-opens
»  fatal error in thread [opensearch[integTest-0][training][T#1]], exiting
»  java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
»       at java.base/java.lang.ClassLoader.defineClass1(Native Method)
»       at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027)
»       at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
»       at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
»       at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
»       at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
»       at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:872)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
»       at java.base/java.lang.ClassLoader.defineClass1(Native Method)
»       at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027)
»       at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
»       at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
»       at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
»       at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
»       at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:872)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
»       at java.base/java.lang.ClassLoader.defineClass1(Native Method)
»       at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027)
»       at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
»       at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
»       at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
»       at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
»       at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
»       at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:872)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
»       at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3515)
»       at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3519)
»       at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2168)
»       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2079)
»       at com.google.common.cache.LocalCache.get(LocalCache.java:4017)
»       at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4898)
»       at org.opensearch.knn.index.memory.NativeMemoryCacheManager.get(NativeMemoryCacheManager.java:441)
»       at org.opensearch.knn.training.TrainingJob.run(TrainingJob.java:118)
»       at org.opensearch.knn.training.TrainingJobRunner.lambda$train$4(TrainingJobRunner.java:133)
»       at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:916)
»       at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
»       at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
»       at java.base/java.lang.Thread.run(Thread.java:1583)
»  Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
»       at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
»       at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:872)
»       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
»       ... 46 more

> Task :integTest FAILED

navneet1v · 2025-09-18T23:57:11Z

@finnegancarroll I tried this change: https://github.com/opensearch-project/k-NN/pull/2878/files and I was able to fix this error but other errors related to 10.3 started popping up. I am fixing them. But WDYT about the fix on build.gradle

finnegancarroll · 2025-09-19T19:01:13Z

I think the problem of InternalFutureFailureAccess failing to load at runtime is due to the class loader hierarchy in OpenSearch. knn depends on transport-grpc so the grpc class loader is responsible for loading failureaccess within knn integ/unit tests, however failureaccess is now compile only in transport-grpc so it is never found. No matter where I place the correct failureaccess jar inside the knn test bundle it doesn't seem to help.

I see integration tests passing locally with:
1 - The core PR @karenyrx opened here cherry picked into my OS distribution.
2 - The lucene bump reverted from my OS distribution.
3 - Force resolve to error_prone_annotations:2.41.0 in build.gradle:

configurations.all {
    resolutionStrategy {
        force 'com.google.errorprone:error_prone_annotations:2.41.0'
    }
}

@navneet1v I think this^ is the only change you need to you build.gradle (lucene changes aside that is).

Signed-off-by: Finn Carroll <[email protected]>

finnegancarroll · 2025-09-19T23:14:24Z

Local testing setup:

Building OS distribution without lucene version bump and with failure access in transport-grpc SPI:
opensearch-project/OpenSearch@1419ecf
opensearch-project/OpenSearch@c56da68

git revert c56da6897e8336398b9fe4187a97c90e42e06024
git cherry-pick 1419ecff568c054379362760d07bb0b3925b365b
./gradlew :distribution:archives:linux-tar:assemble
./gradlew publishToMavenLocal

Testing knn ITs with single local node:

export OS_DIST=<repo-path>OpenSearch/distribution/archives/linux-tar/build/distributions/opensearch-min-3.3.0-SNAPSHOT-linux-x64.tar.gz
./gradlew run \
    -Drepos.mavenLocal \
    -PcustomDistributionUrl=${OS_DIST} \
    --info --stacktrace

# run knn integ tests
./gradlew :integTestRemote \
    -Drepos.mavenLocal \
    -Dtests.rest.cluster=localhost:9200 \
    -Dtests.cluster=localhost:9200 \
    -Dtests.clustername="integTest-0" \
    -Dhttps=false \
    -PnumNodes=1

navneet1v · 2025-09-19T23:47:28Z

I think the problem of InternalFutureFailureAccess failing to load at runtime is due to the class loader hierarchy in OpenSearch. knn depends on transport-grpc so the grpc class loader is responsible for loading failureaccess within knn integ/unit tests, however failureaccess is not compile only in transport-grpc so it is never found. No matter where I place the correct failureaccess jar inside the knn test bundle it doesn't seem to help.

I see integration tests passing locally with: 1 - The core PR @karenyrx opened here cherry picked into my OS distribution. 2 - The lucene bump reverted from my OS distribution. 3 - Force resolve to error_prone_annotations:2.41.0 in build.gradle:
configurations.all {
    resolutionStrategy {
        force 'com.google.errorprone:error_prone_annotations:2.41.0'
    }
}
@navneet1v I think this^ is the only change you need to you build.gradle (lucene changes aside that is).

Yes you are right. Extra ones I have to clean up.

navneet1v · 2025-09-19T23:49:13Z

I am working on complete fix and will raise PR soon. Couple of unit tests are left to be fixed

finnegancarroll marked this pull request as ready for review September 18, 2025 19:41

finnegancarroll requested review from 0ctopus13prime, VijayanB, Vikasht34, heemin32, jmazanec15, junqiu-lei, luyuncheng, martin-gaievski, naveentatikonda, navneet1v, ryanbogan, shatejas and vamshin as code owners September 18, 2025 19:41

finnegancarroll mentioned this pull request Sep 18, 2025

Fix guava and errorprone dependencies #2883

Closed

5 tasks

finnegancarroll mentioned this pull request Sep 19, 2025

Fix KNN build due to Lucene 10.3 upgrade #2878

Merged

5 tasks

Bump error prone annotations 2.41.0 -> 2.26.1.

4a6638e

Signed-off-by: Finn Carroll <[email protected]>

finnegancarroll force-pushed the bump-err-prone branch from ca86f11 to 4a6638e Compare September 19, 2025 19:06

finnegancarroll mentioned this pull request Sep 19, 2025

Add failureaccess dependency to transport-grpc module opensearch-project/OpenSearch#19339

Merged

3 tasks

finnegancarroll closed this Sep 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Force core :server error prone annotations version #2885

Force core :server error prone annotations version #2885

Uh oh!

finnegancarroll commented Sep 18, 2025 •

edited

Loading

Uh oh!

finnegancarroll commented Sep 18, 2025

Uh oh!

navneet1v commented Sep 18, 2025

Uh oh!

finnegancarroll commented Sep 18, 2025

Uh oh!

navneet1v commented Sep 18, 2025

Uh oh!

finnegancarroll commented Sep 19, 2025 •

edited

Loading

Uh oh!

finnegancarroll commented Sep 19, 2025

Uh oh!

navneet1v commented Sep 19, 2025

Uh oh!

navneet1v commented Sep 19, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Force core :server error prone annotations version #2885

Force core :server error prone annotations version #2885

Uh oh!

Conversation

finnegancarroll commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

finnegancarroll commented Sep 18, 2025

Uh oh!

navneet1v commented Sep 18, 2025

Uh oh!

finnegancarroll commented Sep 18, 2025

Uh oh!

navneet1v commented Sep 18, 2025

Uh oh!

finnegancarroll commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

finnegancarroll commented Sep 19, 2025

Uh oh!

navneet1v commented Sep 19, 2025

Uh oh!

navneet1v commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

finnegancarroll commented Sep 18, 2025 •

edited

Loading

finnegancarroll commented Sep 19, 2025 •

edited

Loading

navneet1v commented Sep 19, 2025 •

edited

Loading