Skip to content

Latest commit

 

History

History
359 lines (285 loc) · 14.4 KB

README.md

File metadata and controls

359 lines (285 loc) · 14.4 KB

Java 11+ Build Status Maven Central Code Coverage javadoc (latest release) javadoc (main branch) Issues License Activity

java-compiler-testing

A framework for performing exhaustive integration testing against Java compilers in modern Java libraries, with a focus on full JPMS support.

The Java Compiler Testing API has a number of facilities for assisting in testing anything related to the Java compiler. This includes Javac plugins and JSR-199 annotation processors.

All test cases are designed to be as stateless as possible, with facilities to produce in-memory file systems or using OS-provided temporary directories. All file system mechanisms are complimented with a fluent API that enables writing expressive declarations without unnecessary boilerplate.

Integration test cases can be written to cross-compile against a range of Java compiler versions, with the ability to provide as much or as little configuration detail as you wish. Additionally, APIs can be easily extended to integrate with any other JSR-199-compliant compiler as required.

Compilation results are complimented with a suite of assertion facilities that extend the AssertJ API to assist in writing fluent and human-readable test cases for your code. Each of these assertions comes with specially-developed human-readable error messages and formatting.

Full JUnit5 integration is provided to help streamline the development process.

Features

  • Implements in-memory file management compatible with the NIO Path and FileSystem API, enabling tests to run without write access to the host system, and without awkward resource-cleanup logic.
  • Enables running compilations on combinations of real files, class path resources, in-memory files, JARs, WARs, EARs, ZIP files, etc.
  • Null-safe API (using JSpecify).
  • Tested on multiple existing frameworks including Avaje, Spring, Lombok, MapStruct, ErrorProne, and CheckerFramework.
  • Supports Java 9 JPMS modules.
  • Ability to customise a large assortment of configuration parameters to enable you to test exactly what you need to test.
  • Provides support for javac out of the box, with the ability to support other JSR-199 implementations if desired -- just make use of one of the compiler classes, or make your own!
  • Implements a fully functional JSR-199 Path JavaFileManager and class loading mechanism.
  • Fluent syntax for creating configurations, executing them, and inspecting the results.
  • Integration with AssertJ for fluent assertions on compilation results.
  • Ability to have multiple source roots, just like when using javac normally.
  • Diagnostic reporting includes stack traces, so you can find out exactly what triggered a diagnostic and begin debugging any issues in your applications quickly.

Installation

The project can be found on Maven Central.

<dependency>
  <groupId>org.github.ascopes.jct</groupId>
  <artifactId>java-compiler-testing</artifactId>
  <version>${java-compiler-testing.version}</version>
</dependency>

If you are using Gradle, make sure you enable the Maven Central repositories first, otherwise the dependency will not resolve.

repositories {
  mavenCentral()
}

dependencies {
  testImplementation("io.github.ascopes.jct:java-compiler-testing:$jctVersion")
}

JPMS

If your tests make use of JPMS (i.e. they have a module-info.java somewhere), then you will want to add a requirement for this module like so:

module my.tests {
  ...
  requires org.assertj.core;
  requires io.github.ascopes.jct;
  ...
}

JUnit5 integration

While this library provides JUnit5 support and extensions, it does not provide JUnit5 as a direct dependency. You should ensure you have a working JUnit5 configuration in your project prior to including this library. This includes junit-jupiter and junit-jupiter-params.

For Maven, it is also suggested to ensure you are using maven-surefire-plugin (or maven-failsafe-plugin) on version 3.0.0-M1 or newer. This ensures that JPMS is handled correctly.

If making use of JPMS, you should include the following in your module-info.java:

open module my.tests {
  ...
  requires org.assertj.core;
  requires io.github.ascopes.jct;
  requires transitive org.junit.jupiter;
  requires org.junit.jupiter.params;
  ...
}

Examples

In-memory code, using RAM disks for source directories

@DisplayName("Example tests")
@ExtendWith(JctExtension.class)
class ExampleTest {
  
  @Managed
  Workspace workspace;

  @DisplayName("I can compile a Hello World application")
  @JavacCompilerTest
  void canCompileHelloWorld(JctCompiler compiler) {
    // Given
    workspace
        .createSourcePathPackage()
        .createFile("org/example/HelloWorld.java").withContents("""
            package org.example;
                
            @Data
            public class HelloWorld {
              public static void main(String[] args) {
                System.out.println("Hello, World!");
              }
            }
            """
        );

    // When
    var compilation = compiler.compile(workspace);

    // Then
    assertThatCompilation(compilation)
        .isSuccessfulWithoutWarnings();

    assertThatCompilation(compilation)
        .classOutputPackages()
        .fileExists("org/example/HelloWorld.class")
        .isNotEmptyFile();
  }
}

Compiling and testing with a custom annotation processor

import static io.github.ascopes.jct.assertions.JctAssertions.assertThatCompilation;

import io.github.ascopes.jct.compilers.JctCompiler;
import io.github.ascopes.jct.junit.JavacCompilerTest;
import io.github.ascopes.jct.junit.JctExtension;
import io.github.ascopes.jct.junit.Managed;
import io.github.ascopes.jct.workspaces.Workspaces;
import org.example.processor.JsonSchemaAnnotationProcessor;
import org.skyscreamer.jsonassert.JSONAssert;

@ExtendWith(JctExtension.class)
class JsonSchemaAnnotationProcessorTest {
  
  @Managed
  Workspace workspace;

  @JavacCompilerTest(minVersion = 11, maxVersion = 21)
  void theJsonSchemaIsCreatedFromTheInputCode(JctCompiler compiler) {
    // Given
    workspace
        .createSourcePathPackage()
        .createDirectory("org", "example", "tests")
        .copyContentsFrom("src", "test", "resources", "code", "schematest");

    // When
    var compilation = compiler
        .addAnnotationProcessors(new JsonSchemaAnnotationProcessor())
        .addAnnotationProcessorOptions("jsonschema.verbose=true")
        .failOnWarnings(true)
        .showDeprecationWarnings(true)
        .compile(workspace);

    // Then
    assertThatCompilation(compilation)
        .isSuccessfulWithoutWarnings();

    assertThatCompilation(compilation)
        .diagnostics().notes().singleElement()
        .message().isEqualTo(
            "Creating JSON schema in Java %s for package org.example.tests",
            compiler.getRelease()
        );

    assertThatCompilation(compilation)
        .classOutputPackages()
        .fileExists("json-schemas", "UserSchema.json").contents()
        .isNotEmpty()
        .satisfies(contents -> JSONAssert.assertEquals(...));
  }
}

Compiling multi-module sources

The following shows an example of compiling a multi-module style application with JPMS support, running the Lombok annotation processor over the input. This assumes that the Lombok JAR is already on the classpath for the JUnit test runner (e.g. is a test dependency in your project).

You will want to make sure you do not attempt to target anything older than Java 9 in this case, since module support was only introduced in Java 9.

@DisplayName("Example tests")
class ExampleTest {

  @DisplayName("I can compile a module that is using Lombok")
  @JavacCompilerTest(minVersion = 9)
  void canCompileModuleUsingLombok(JctCompiler compiler) {
    try (var workspace = Workspaces.newWorkspace()) {
      // Given
      workspace
          .createSourcePathModule("hello.world")
          .createFile("org/example/Message.java").withContents("""
                package org.example;

                import lombok.Data;
                import lombok.NonNull;

                @Data
                public class Message {
                  @NonNull
                  private final String content;
                }
              """
          )
          .and().createFile("org/example/Main.java").withContents("""
                package org.example;

                public class Main {
                  public static void main(String[] args) {
                    for (var arg : args) {
                      var message = new Message(arg);
                      System.out.println(arg);
                    }
                  }
                }
              """
          )
          .and().createFile("module-info.java").withContents("""
                module hello.world {
                  requires java.base;
                  requires static lombok;
                }
              """
          );

      // When
      var compilation = compiler.compile(workspace);

      // Then
      assertThatCompilation(compilation)
          .isSuccessfulWithoutWarnings();

      assertThatCompilation(compilation)
          .classOutputPackages()
          .fileExists("org/example/Message.class")
          .isNotEmptyFile();
    }
  }
}

Tips to improve build speeds

Running a Java Compiler and maintaining a virtual file system is not the simplest thing to achieve internally. As a result, some tests may take a few hundred milliseconds to execute in some cases, especially when the JVM is warming up. There are a few things you can do to ensure that tests are as snappy as possible, however.

Parallel test runners

JUnit5 has the ability to configure tests to run in parallel.

You can configure this by creating a file in your test resources named junit-platform.properties and add the following content to it.

junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.classes.default=SAME_THREAD
junit.jupiter.execution.parallel.mode.default=CONCURRENT

Within build systems like Maven, you can also enable your builds to run in parallel where desired. In the case of Maven, you can pass -T1C on the command line to make it run one parallel operation on each CPU core.

JVM flags

You can provide JVM flags to the runtime that executes your test packs. For Maven Surefire, you do this by defining an <argLine/> attribute in your <properties/> with a string value holding the flags you wish to use.

  1. Enforce level-1 tiered compilation - this will prevent the JVM wasting time performing more complicated JIT compilation passes over your code when it usually does not provide much benefit on short-lived code. You can pass -XX:+TieredCompilation -XX:TieredStopAtLevel=1 to set this up. Enabling this in the JCT builds reduced the overall build time by around 20 seconds.
  2. Use the ZGC - the ZGC will reduce lag when performing garbage collection on code that has a high churn of objects. On Java 11, the ZGC is an experimental feature, which needs to be enabled with -XX:+UnlockExperimentalOptions -XX:+UseZGC. On Java 17, you just need to pass -XX:+UseZGC alone.

Third-party compiler support

The base classes to provide third party compiler integrations are made public in this API so you can extend them.

To integrate with your chosen JSR-199 compatible compiler, extend the AbstractJctCompiler class and override anything you need to tweak.

The call to compile will return a JctCompilation object. This is already defined for you. All you need to provide in your compiler class is:

ECJ (Eclipse Java Compiler)

While ECJ supports the same interfaces as Javac that are used to call the compiler from this library, eclipse-jdt/eclipse.jdt.core#1153 means we will not get functional Java 11 support going forwards.

A number of issues were found while developing ascopes/java-compiler-testing#163 with ECJ which prevents many features such as JPMS support from working correctly (eclipse-jdt/eclipse.jdt.core#958).

Since it is unlikely these issues will be addressed in the near future, support for ECJ has been shelved for the foreseeable future.