Pro Plugin Development

[sormuras]

I would like to contribute the following two plugins:

• JUnit-Platform-Console-Standalone-Plugin
• Google-Java-Format-Plugin

Both underlying tools are packaged as executable JARs, which can be call via the command line. That's why I think a more general and configurable Download-Dependencies-And-Execute-Plugin would be more useful and be used by other external tools as well. Like checkstyle, coverage, etc.

An usage outline could read like:

set("junit.dependencies", list("junit-platform-console-standalone:1.0.0-SNAPSHOT"))

...

run(... "compiler", "packager", "Download-Dependencies-And-Execute-Plugin(junit)")

Do you think it is worth the effort? Currently, I'm using wget/java commands in the .travis.yml build script.

[forax]

I've tried to not used the command line to run plugins in pro for several reasons.
A first reason is speed, pro is currently quite fast (compared to other build tools) because from the VM point of view, pro is one Java program so you do not pay the time to ramp up a new VM each time you run a plugin.
A second reason is that pro should offer a programmatic API, hence the name, with useful defaults if your abstraction, is just a Java program or a shell program, you can not provide such API with an implicit configuration.
A third reason, is that i want to be able to re-play a build so if you take a look, pro is quite big because it uses its own JDK with its own dependencies, etc. So when you have a version of pro + a specific configuration file should always build the same artifacts whatever is installed .

so i prefer you to develop plugins (a plugin is usually 1 class and 2 interfaces), one plugin can use a jar that comes from Maven Central, call the main method of the main class of that jar, and it will provide a nice experience out of the box, even if it means that it's less flexible.

[sormuras]

I totally agree.

Is there already an example plugin or built-in one that I can use to learn how to write such a plugin that depends on an external jar?

[sormuras]

I started an empty "junitconsole" plugin and need to (auto) fix the JAR in deps/. Locally, I have added deps/junit-platform-console-standalone-1.0.0-20170331.213831-86.jar (soon M4 from Maven Central) which does not contain a module-info.class, yet.

See the skeleton at master...sormuras:plugin_junitconsole

Edit: with junit-team/junit-framework#761 fixed, the bootstrap module-fixer works like a charm!

[sormuras]

Next task:

[modulefixer] multiple modules [org.junit.jupiter.api, junit.platform.console.standalone@1.0.0-SNAPSHOT] contains package org/junit/jupiter/api/extension
[modulefixer] multiple modules [junit.platform.console.standalone@1.0.0-SNAPSHOT, org.opentest4j] contains package org/opentest4j
[modulefixer] multiple modules [org.junit.jupiter.api, junit.platform.console.standalone@1.0.0-SNAPSHOT] contains package org/junit/jupiter/api
[modulefixer] multiple modules [junit.platform.console.standalone@1.0.0-SNAPSHOT, org.junit.platform.commons] contains package org/junit/platform/commons/annotation
[modulefixer] multiple modules [org.junit.jupiter.api, junit.platform.console.standalone@1.0.0-SNAPSHOT] contains package org/junit/jupiter/api/function
[modulefixer] multiple modules [junit.platform.console.standalone@1.0.0-SNAPSHOT, org.junit.platform.commons] contains package org/junit/platform/commons/meta
[modulefixer] multiple modules [junit.platform.console.standalone@1.0.0-SNAPSHOT, org.junit.platform.commons] contains package org/junit/platform/commons
[modulefixer] multiple modules [junit.platform.console.standalone@1.0.0-SNAPSHOT, org.junit.platform.commons] contains package org/junit/platform/commons/util
[pro] FAILED !        elapsed time 316 ms

How to resolve them?

[forax]

In src/main/java/com.github.forax.pro.plugin.junitconsole/module-info.java,
i believe, you do not need to 'opens' com.github.forax.pro.plugin.junitconsole given that you provide your plugin as a com.github.forax.pro.api.Plugin.

The error messages are due to the fact that you have 'split packages', packages that exists in several modules, this is forbidden by Java 9. Only one module can own a package. Otherwise, it means that you can inject any class to any existing package bypassing the security.

There is only one way to solve that, is to move classes between modules to have packages only defined by one module. As a workaround, you can also merge several jars to one bigger jar that will be considered as one module.

The module fixermay solve that for you by merging the jars automatically to create big fat module.
But that seems a bit to automagic for me

[sormuras]

Understood.

The split packages result from the "junit-fat-all-standalone.jar" (it contains JUnit 4+5 and dependencies) being visible to the actual test compilation process, which of course refers to the API package required by the test code. Why is the auto-fixed "junit-fat-all-standalone.jar" module visible to user-space (test) code?

[forax]

It's a known limitation of pro,
pro creates it's own JDK, and all dependencies are considered as a root modules, if you prefer all dependencies of a plugin is considered as visible as java.base :(

I should try to sandbox each plugin in its own layer or at least, separate the plugins from the rest of the runtime .
I will try to come up with a solution for that soon.

[sormuras]

I see. Thanks for the explanation.

Maybe, test execution doesn't have to be part of the pro build. The external tool approach using wget+java works: https://travis-ci.org/sormuras/beethoven/builds/217367445#L345

[sormuras]

Hm, when pro provides (pun intended) all its own dependencies to the projects, how can I configure the resolver/compiler to use the provided modules, like "org.json" or here "junit-platform-console-standalone"?

At the moment, I'm using this "build.pro" file -- but the resolver always tries to reload & fix the junit JAR:

set("compiler.rawArguments", list( "-encoding", "UTF8" ))
run("resolver", "modulefixer", "compiler", "packager", "junitconsole")

Along with a minimal test "module-info.java", requiring exactly the same module as the junitconsole plugin:

module de.sormuras.beethoven {
  requires jdk.compiler;
  requires jdk.scripting.nashorn;
  requires java.desktop;

  // requires org.junit.jupiter.api;
  // requires org.junit.platform.commons;
  // requires org.opentest4j;

  requires junit.platform.console.standalone; // provided by pro plugin "junitconsole"
}

The junitconsole plugin is still a skeleton w/ the fixed JAR included: master...sormuras:plugin_junitconsole

[forax]

The name of a plain old jar (not a modular jar) is extracted from the name of the jar or from the an attribute ModuleName in the jar Manifest by the JDK before being fixed by the module fixer.
The module fixer do not change the name of the module.

So apart if your jar is not a modular jar (has a module-info.class), either you can rename the jar (use '.' instead of '-'), or you add the attribute "ModuleName: " with the right name in your jar.

[sormuras]

[...] add the attribute "ModuleName: " with the right name in your jar.

Interesting. Is this attribute an official feature or a good idea implemented in the module fixer?

[forax]

It's an official feature, but i've just checked and it's not yet integrated :(

What is the name of the jar exactly ?

[sormuras]

junit-platform-console-standalone-1.0.0-M4.jar

http://central.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.0.0-M4/junit-platform-console-standalone-1.0.0-M4.jar

I renamed it to deps/junit.platform.console.standalone-1.0.0_M4.jar and build pro. But it doesn't show up in the build logs. Like aether and the rest. I guess, all modules from "deps" are merged into target/pro/lib/modules, right?

[sormuras]

Looks like, I got it running now.

I see the log statements like:

...
[compiler] copy src\test\resources to target\test\exploded
[junitconsole] config {
...

}
|  State engine terminated.
|  Restore definitions with: /reload -restore
->

...and the the JShell prompt is blinking. I need to type "/exit" to quit. I guess, I now may start implementing the plugin now.

[sormuras]

Is the initial work here master...sormuras:plugin_junitconsole doing it in the right direction?