native-image : @Delete expects all dependencies of target class to be available?

[jaikiran]

Imagine this original class:

package org.myapp;

public class FooBar {
   public void doSomething(org.otherapp.SomeThirdPartyLibClass arg) {
     ....
   }
}

Now imagine this usage of @Delete:

import com.oracle.svm.core.annotate.Delete;
import com.oracle.svm.core.annotate.TargetClass;

@TargetClass(className = "org.myapp.FooBar")
@Delete
public final class Delete_FooBar {
}

So there's a org.myapp.FooBar which I wanted deleted from the native image and I have a @Delete which tries to achieve that. The org.myapp.FooBar has fields/methods which use types that are part of other (third party) libraries. Those libraries aren't present in the classpath of the native-image generation command. When I run the native-image command which looks something like this:

graalvm/graalvm-ce-19.2.0.1/Contents/Home/bin/native-image 
--report-unsupported-elements-at-runtime --initialize-at-build-time= 
-H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime 
-jar myfoobarapp.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0
 -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree 
 -H:Log=registerResource: -H:-AddAllCharsets -H:EnableURLProtocols=http 
 -H:-JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace

This runs into an exception:

Fatal error: java.lang.NoClassDefFoundError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
	at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:113)
Caused by: java.lang.NoClassDefFoundError: javax/security/jacc/PolicyContextException
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.getDeclaredMethods(Class.java:1975)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:437)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:270)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:230)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:875)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:824)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:524)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: org.otherapp.SomeThirdPartyLibClass
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 15 more
Error: Image build request failed with exit status 1

So it looks like the usage of @Delete expects the classpath to be complete with all the necessary dependencies, even though the target class is really being removed from the native-image? Is there a way the @Delete can work without expecting the whole dependency graph to be avaiable to native image generation?

I'm on Graal VM version 19.2.0.1.

[jaikiran]

Interestingly, this usage fails with that exception only when --report-unsupported-elements-at-runtime option is used during native image generation. If that option isn't used then native image generation succeeds without any issues.

So, IMO, this really looks like an bug in the implementation of how this is handled in https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/substitute/AnnotationSubstitutionProcessor.java#L432

Here's an sample application (with a README) which reproduces this issue https://github.com/jaikiran/substratevm-test-deletion

[christianwimmer]

Unfortunately, there is no straightforward fix for this problem (otherwise we would of course just fix it).

I think there is some misconception about --report-unsupported-elements-at-runtime and @Delete:
We introduced the option --report-unsupported-elements-at-runtime so that developers can delay thinking about some Native Image compatibility issues and make quick progress getting a first image built. But that option fundamentally hides problems at image build time that you actually want to have checked. So --report-unsupported-elements-at-runtime should not be used for any images that you want to use in production.
So if you mark something as @Delete (which is of course an internal annotation and not part of our API), you are telling the analysis that it is an error if the annotated code is reachable. But then you want this error reported at run time?

[ppalaga]

So if you mark something as @delete (which is of course an internal annotation and not part of our API), you are telling the analysis that it is an error if the annotated code is reachable. But then you want this error reported at run time?

Not sure I follow this. Yes, it makes no sense to @Delete and report on that particular class on runtime. I do not want that at all. What I want is --report-unsupported-elements-at-runtime to work for other classes that I have not deleted. Combining --report-unsupported-elements-at-runtime with @Delete should not break the native compilation.

[loicmathieu]

@christianwimmer so maybe there is a broad misuse of --report-unsupported-elements-at-runtime because library authors like Spring Native and Google Cloud Native (also shaded from an upstream library it seems) uses it inside there native-image.properties file. I agree that this should be a developer's only flag but the usage is real.

Maybe we need a --disable-report-unsupported-elements-at-runtime to force ignoring the flag at runtime when putting inside a native-image.properties file.

[dzou]

I think there is some misconception about --report-unsupported-elements-at-runtime and @Delete:
We introduced the option --report-unsupported-elements-at-runtime so that developers can delay thinking about some Native Image compatibility issues and make quick progress getting a first image built. But that option fundamentally hides problems at image build time that you actually want to have checked. So --report-unsupported-elements-at-runtime should not be used for any images that you want to use in production.
So if you mark something as @Delete (which is of course an internal annotation and not part of our API), you are telling the analysis that it is an error if the annotated code is reachable. But then you want this error reported at run time?

Hey, I'm curious if we could revisit this discussion.

While I understand that --report-unsupported-elements-at-runtime should be for dev-only, I don't think the policy works in practice. This is because in the early stages of GraalVM adoption it is hard to get everything working at once without it for many library maintainers. A library will go through alpha or beta stages before a GA release of GraalVM support, and during the experimental stages I don't think it's unreasonable to have the flag.

This is why the setting is enabled by default in Spring Native (beta) and also for Google libraries (pre-GA).

But I still think annotations for @Delete should work though. I basically think it goes back to understanding how long it takes complex libraries/SDKs to adopt GraalVM; if this fact is acknowledged, then it seems more reasonable to have things compatible with --report-unsupported-elements-at-runtime.

[christianwimmer]

I think the only feasible option is to switch to something better than @Delete to cut off code that should be unreachable in a native image. Something like #3225

[zakkak]

Fixed by 30ae9d3 in GraalVM for JDK 23

[ppalaga]

Fixed by 30ae9d3 in GraalVM for JDK 23

So @Delete can now be used to mark optional code that is not available?

[zakkak]

So @Delete can now be used to mark optional code that is not available?

Yes, but you will need the upcoming GraalVM for JDK 23 (expected to be released next week).

I have also created a backport PR to bring this in GraalVM for JDK 21 as well, see graalvm/graalvm-community-jdk21u#22. Which might land in the next CPU release (mid or late October 2024)

[ppalaga]

Great to hear that! I hope, we will be able to remove all the method stubs we created to make GraalVM happy.

[zakkak]

Great to hear that! I hope, we will be able to remove all the method stubs we created to make GraalVM happy.

@ppalaga I might have misunderstood your question. You won't be able to "mark optional code that is not available". After all what's the point of deleting a non-existent class. But you will be able to delete classes that are available, but reference unavailable code.

As a reminder there is #3225 as well.

If you can point me to a specific case I could see if this fix affects it or not.

[ppalaga]

But you will be able to delete classes that are available, but reference unavailable code.

That might be the case from #3225 (comment)

I'll try the improved @Delete once the new GraalVM version is out.