Skip to content

Make it easier to test against multiple versions of the JDK #1072

@RyanGlScott

Description

@RyanGlScott

In my work on #1046, I briefly considered extending saw-script's CI so that in addition to testing JDK 8, it would also test a newer JDK version (e.g., JDK 15) that exercised some new code paths in saw-script. Ultimately, I didn't pursue this direction too deeply for two reasons:

  1. Support for modern JDKs is still experimental in light of crucible-jvm: Investigate bugs when verifying JDK 9+ code that uses String crucible#641.
  2. The naïve way of testing multiple JDKs—namely, doubling the number of jobs such that half of them test JDK 8 and the other half test JDK 15—would result in a lot of wasted work, as we only really need to re-run the parts of CI that exercise Java–specific code paths. For example, we really only ought to run LLVM-related tests once overall, not once per JDK version.

GaloisInc/crucible#641 tracks (1), so this issue exists to track (2). The way I see it, we should do something like this:

  • Investigate if setup-java makes it possible to simultaneously have multiple versions of the JDK installed side by side, allowing one to switch between one version or the other. My hope is that setup-java allows for something akin to update-java-alternatives, but I'm not sure if that's the case off the top of my head.
  • Clearly indicate in the test suite which tests use Java and which do not so that we can parameterize the Java-specific tests by the particular JDK version being tested against. That way, we need only re-run the Java specific tests if we support multiple JDKs, and we can avoid re-running other tests such as LLVM-specific tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    subsystem: crucible-jvmIssues related to Java verification with crucible-jvmtooling: test infrastructureIssues involving test infrastructure or test execution, or making SAW more testabletype: feature requestIssues requesting a new feature or capability

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions