Native images throw MissingResourceRegistrationError despite the resource being actually registered (but not present)

[zakkak]

Describe the issue

While working on quarkusio/quarkus#36378 I came across the following issue.
When using -H:ThrowMissingRegistrationErrors= -H:MissingRegistrationReportingMode=Exit at build time the resulting native image throws an exception if it tries to access a resource that is not available but was registered.

The native image should not throw a missing registration error in that case, as the resource is actually registered (in resource-config.json) despite not existing.

cc @vjovanov

Steps to reproduce the issue

cd /tmp
git clone --branch 2024-06-04-missing-metadata-error https://github.com/zakkak/issue-reproducers reproducer
cd reproducer
mvn package
java -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image -jar target/reproducer-1.0-SNAPSHOT.jar
mvn package
native-image -jar target/reproducer-1.0-SNAPSHOT.jar -H:+UnlockExperimentalVMOptions -H:ThrowMissingRegistrationErrors= -H:MissingRegistrationReportingMode=Exit
./reproducer-1.0-SNAPSHOT

This results in:

Hello there!
com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path 

   missing-test.txt

 without it being registered as reachable. Add it to the resource metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#resources-and-resource-bundles for help
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.resources.MissingResourceRegistrationUtils.missingResource(MissingResourceRegistrationUtils.java:43)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.Resources.createURLs(Resources.java:459)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.Resources.createURL(Resources.java:384)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.Resources.createURL(Resources.java:376)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.ResourcesHelper.nameToResourceURL(ResourcesHelper.java:103)
        at [email protected]/jdk.internal.loader.BootLoader.findResource(BootLoader.java:631)
        at [email protected]/java.lang.ClassLoader.getResource(ClassLoader.java:1409)
        at [email protected]/java.lang.ClassLoader.getResource(ClassLoader.java:1407)
        at [email protected]/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1750)
        at Main.main(Main.java:15)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

while I would expect it to result in:

Hello there!
Resource missing-test.txt not found!

Describe GraalVM and your environment:

• GraalVM version: GraalVM CE 21.0.2+13.1
• JDK major version: 21
• OS: Fedora 40
• Architecture: AMD64

More context:

My understanding is that the generated native images rely on the embedded resources map to throw these errors instead of a "registration" map. So if we register *.txt the native image will only know about the txt files that ended up being embedded in the native image (i.e. they were on the classpath), the current implementation can't understand that any *.txt is actually registered...

This is an issue for applications trying to load a resource that was not available at build time but can handle that case, e.g. trying to load a configuration and falling back on the default if it doesn't exist...

[fernando-valdez]

Thanks for reporting this issue. We will take care of it.

[Karm]

@zakkak I am having trouble to reproduce it running latest native-image with my code snippet.

Directory:

.
./Main.java
./myfile-1.txt
./myfile-2.txt

Files just contain "Hello from file 1" etc.

Code:

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;

public class Main {
    public static void main(String[] args) throws IOException {
        for (String s : new String[] { "myfile-1.txt", "myfile-2.txt", "myfile-3.txt" }) {
            try (final InputStream is = Objects.requireNonNull(Main.class.getResourceAsStream("/" + s),
                    "File " + s + " not found.")) {
                System.out.print(new String(is.readAllBytes(), StandardCharsets.US_ASCII));
            } catch (NullPointerException e) {
                System.out.println(e.getMessage());
            }
        }
    }
}

🔴 FAIL GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30)

$ javac Main.java

$ java -agentlib:native-image-agent=config-output-dir=agent-stuff Main
Hello from file 1
Hello from file 2
File myfile-3.txt not found.

$ cat agent-stuff/resource-config.json 
{
  "resources":{
  "includes":[{
    "pattern":"\\Qmyfile-1.txt\\E"
  }, {
    "pattern":"\\Qmyfile-2.txt\\E"
  }, {
    "pattern":"\\Qmyfile-3.txt\\E"
  }]},
  "bundles":[]
}


$ native-image -H:+UnlockExperimentalVMOptions -H:ThrowMissingRegistrationErrors= -H:ConfigurationFileDirectories=agent-stuff Main main

$ ./main 
Hello from file 1
Hello from file 2
Exception in thread "main" com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path myfile-3.txt without it being registered as reachable. Add it to the resource metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#resources-and-resource-bundles for help
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.resources.MissingResourceRegistrationUtils.missingResource(MissingResourceRegistrationUtils.java:41)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.Resources.createInputStream(Resources.java:418)
        at [email protected]/java.lang.Class.getResourceAsStream(DynamicHub.java:821)
        at Main.main(Main.java:9)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

🟢 PASS GraalVM CE 22.0.1+8.1 (build 22.0.1+8-jvmci-b01)

$ rm -rf agent-stuff

$ java -agentlib:native-image-agent=config-output-dir=agent-stuff Main
Hello from file 1
Hello from file 2
File myfile-3.txt not found.

$ cat agent-stuff/resource-config.json 
{
  "resources":{
  "includes":[{
    "pattern":"\\Qmyfile-1.txt\\E"
  }, {
    "pattern":"\\Qmyfile-2.txt\\E"
  }, {
    "pattern":"\\Qmyfile-3.txt\\E"
  }]},
  "bundles":[]
}

$ native-image -H:+UnlockExperimentalVMOptions -H:ThrowMissingRegistrationErrors= -H:ConfigurationFileDirectories=agent-stuff Main main

$ ./main 
Hello from file 1
Hello from file 2
File myfile-3.txt not found.

🟢 PASS (Latest) Mandrel-24.2.0-dev6eefc9c7aae9 (build 24-beta+3-ea)

$ rm -rf agent-stuff

$ java -agentlib:native-image-agent=config-output-dir=agent-stuff Main

$ cat agent-stuff/reachability-metadata.json 
{
  "reflection":[
  ],
  "resources":[{
    "glob":"myfile-1.txt"
  },{
    "glob":"myfile-2.txt"
  },{
    "glob":"myfile-3.txt"
  }],
  "bundles":[],
  "serialization":[
  ],
  "jni":[
    {
      "type":"sun.launcher.LauncherHelper",
      "fields":[{"name":"isStaticMain"},{"name":"noArgMain"}]
    }
  ]
}

$ native-image -H:+UnlockExperimentalVMOptions -H:ThrowMissingRegistrationErrors= -H:ConfigurationFileDirectories=agent-stuff Main main

$ ./main
Hello from file 1
Hello from file 2
File myfile-3.txt not found.

[zakkak]

You are right, I probably mixed up the versions when reporting this. Updating the description accordingly. Thanks!

Update: On the other hand the logs clearly show 22.0.1, perhaps I built an older graal version using JDK 22...

[zakkak]

The issue is fixed by backporting 2368a2f (part of #7827 which seems to include a number of other fixes probably worth backporting)