Periodically run all tests with the latest JUnit to spot potentially missing classes for buildtime init

[dnestoro]

After we refactored our JUnit feature, there is no need for build-time initialization of any JUnit class, for JDKs newer than JDK21. However, for JDKs <= 21 we still need to maintain list of classes that should be explicitly marked for build-time init.

In order to react on time, we can set a periodic GitHub job that would run all our JUnit tests with:

• latest available JUnit - either with a pre-release or snapshot release
• JDK 17 and 21 (as this check is only required for JDKs <= 21)

If we see any failure, related to the missing build time initialization, we should update the list.

[dnestoro]

@marcphilipp @sbrannen would it be possible to implement similar thing on JUnit side as well? Something like a periodic (or on demand) GitHub action, that would run some set of your tests with the latest Native Build Tools and GraalVM for JDK 21 (for example)?

That way we will spot if we should update initailize-at-buildtime list before every JUnit release.

[sbrannen]

We already have a graalvm-starter project that should be tested with any full build for JUnit.

There's also a list of types that need to be added to support JUnit 6.0 in there.

https://github.com/junit-team/junit-framework/blob/6c411cc703990daffe8a7a09f8d23cee28dc1e2b/platform-tooling-support-tests/projects/graalvm-starter/build.gradle.kts#L30-L39

And the NBT version is managed here.

https://github.com/junit-team/junit-framework/blob/6c411cc703990daffe8a7a09f8d23cee28dc1e2b/platform-tooling-support-tests/projects/graalvm-starter/settings.gradle.kts#L3-L4

So, perhaps that could be "parameterized" to test against different NBT versions.

@marcphilipp, thoughts?

[dnestoro]

@sbrannen Great! In that case, do you think it is necessary at all to have tests (for this purpose) on our side?

So, perhaps that could be "parameterized" to test against different NBT versions.

I think it would be great if you can somehow always pick up the latest NBT version (so that you always use up-to-date list of classes for build-time init)

[marcphilipp]

I think it would be great if you can somehow always pick up the latest NBT version (so that you always use up-to-date list of classes for build-time init)

We use Renovate to keep that version up to date so I think we should be all set. Since the test is quite slow, I'd be hesitant to run it against different GraalVM or NBT versions.

as this check is only required for JDKs <= 21

I actually changed it to run against GraalVM JDK 25 this morning and had to add a class to it. Why is that? 🤔

There's also a list of types that need to be added to support JUnit 6.0 in there.

junit-team/junit-framework@6c411cc/platform-tooling-support-tests/projects/graalvm-starter/build.gradle.kts#L30-L39

There's an open issue on our side as a reminder to send a PR before 6.0 is released. I'll submit one soon, probably next week.

• Add new types to GraalVM Native Build Tools' list of initialize-at-build-time classes junit-team/junit-framework#4582

In that case, do you think it is necessary at all to have tests (for this purpose) on our side?

Probably not, unless you want to have coverage for more/different versions.

[dnestoro]

I actually changed it to run against GraalVM JDK 25 this morning and had to add a class to it. Why is that?

That should not be possible. Can you share minimal reproducer for that? Can you observe the same behavior with GraalVM JDK 23? I've run our JUnit tests with GraalVM JDK 25 and everything worked fine, so I am wondering what caused that issue.

[marcphilipp]

You're right. It only occurs when I pass some classes (those necessary for <= 21) via --initialize-at-build-time.

[marcphilipp]

Related PR:

• Add new JUnit 5.14/6.0 types to initialize-at-build-time classes #771

[dnestoro]

Closing this issue as it is implemented on JUnit side. We will discuss possible testing against multiple versions later