bnd-maven-plugin ConcurrentModificationException on latest OpenJDK 15 EA

[dbwiddis]

I've been using the bnd-maven-plugin for a long time with no issues and have not made any recent configuration changes. Beginning tonight, the Travis CI build is failing on the OpenJDK15 (early access) build with this error:

[ERROR] Failed to execute goal biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process (default) on project oshi-core: bnd error: null: ConcurrentModificationException

The Java version used is:

openjdk version "15-ea" 2020-09-15
OpenJDK Runtime Environment (build 15-ea+18-776)
OpenJDK 64-Bit Server VM (build 15-ea+18-776, mixed mode, sharing)

Link to failing Travis CI job: https://travis-ci.org/github/oshi/oshi/jobs/673288964

It worked as recently as a two days ago on a slightly earlier EA build:

openjdk version "15-ea" 2020-09-15
OpenJDK Runtime Environment (build 15-ea+17-717)
OpenJDK 64-Bit Server VM (build 15-ea+17-717, mixed mode, sharing)

Link to passing Travis CI job from 2 days ago: https://travis-ci.org/github/oshi/oshi/jobs/672568551

Other relevant version strings:

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /home/travis/.m2/wrapper/dists/apache-maven-3.6.3-bin/1iopthnavndlasol9gbrbg6bf2/apache-maven-3.6.3
Java version: 15-ea, vendor: Oracle Corporation, runtime: /home/travis/openjdk15
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-1055-gcp", arch: "amd64", family: "unix"

Relevant portions of my POM configuration:

<properties>
  <bnd-maven-plugin.version>5.0.1</bnd-maven-plugin.version>
</properties>

<build>
  <pluginManagement>
    <plugins>
      <plugin>
        <groupId>biz.aQute.bnd</groupId>
        <artifactId>bnd-maven-plugin</artifactId>
        <version>${bnd-maven-plugin.version}</version>
        <configuration>
          <bnd><![CDATA[
          Export-Package: oshi.*;-noimport:=true;-split-package:=merge-first
          Bundle-SymbolicName: ${project.groupId}.${project.artifactId}
          -snapshot: SNAPSHOT
        ]]></bnd>
        </configuration>
      </plugin>
    </plugins>
  </pluginManagement>
  <plugins>
    <plugin>
      <groupId>biz.aQute.bnd</groupId>
      <artifactId>bnd-maven-plugin</artifactId>
      <executions>
        <execution>
          <goals>
            <goal>bnd-process</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Happy to help troubleshoot but I have no idea where to even start!

[rotty3000]

Would it be possible for you to re-run the build with -X so that you could give us the full stacktrace?

Thanks for reporting!

[dbwiddis]

Figuring how to add the -X to the Travis install script was an adventure, but I finally got it. :)

Link to the build and full log is here: https://travis-ci.org/github/oshi/oshi/jobs/673301183

Relevant (?) trace:

[INFO] --- bnd-maven-plugin:5.0.1:bnd-process (default) @ oshi-core ---
[DEBUG] Configuring mojo biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process from plugin realm ClassRealm[plugin>biz.aQute.bnd:bnd-maven-plugin:5.0.1, parent: java.net.URLClassLoader@37918c79]
[DEBUG] Configuring mojo 'biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process' with basic configurator -->
[DEBUG]   (f) bnd = Export-Package: oshi.*;-noimport:=true;-split-package:=merge-first
						Bundle-SymbolicName: com.github.oshi.oshi-core
						-snapshot: SNAPSHOT
[DEBUG]   (f) bndfile = bnd.bnd
[DEBUG]   (f) classesDir = /home/travis/build/oshi/oshi/oshi-core/target/classes
[DEBUG]   (f) includeClassesDir = true
[DEBUG]   (f) manifestPath = /home/travis/build/oshi/oshi/oshi-core/target/classes/META-INF/MANIFEST.MF
[DEBUG]   (f) mojoExecution = biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process {execution: default}
[DEBUG]   (f) outputDir = /home/travis/build/oshi/oshi/oshi-core/target/classes
[DEBUG]   (f) packagingTypes = [jar, war]
[DEBUG]   (f) project = MavenProject: com.github.oshi:oshi-core:5.0.0-SNAPSHOT @ /home/travis/build/oshi/oshi/oshi-core/pom.xml
[DEBUG]   (f) resources = [Resource {targetPath: null, filtering: false, FileSet {directory: /home/travis/build/oshi/oshi/oshi-core/src/main/resources, PatternSet [includes: {}, excludes: {}]}}]
[DEBUG]   (f) settings = org.apache.maven.execution.SettingsAdapter@6993c8df
[DEBUG]   (f) skip = false
[DEBUG]   (f) sourceDir = /home/travis/build/oshi/oshi/oshi-core/src/main/java
[DEBUG]   (f) targetDir = /home/travis/build/oshi/oshi/oshi-core/target
[DEBUG]   (f) warOutputDir = /home/travis/build/oshi/oshi/oshi-core/target/oshi-core-5.0.0-SNAPSHOT
[DEBUG] -- end configuration --
[DEBUG] loading bnd properties from bnd element in pom: MavenProject: com.github.oshi:oshi-parent:5.0.0-SNAPSHOT @ /home/travis/build/oshi/oshi/pom.xml
[DEBUG] loading bnd properties from bnd element in pom: MavenProject: com.github.oshi:oshi-core:5.0.0-SNAPSHOT @ /home/travis/build/oshi/oshi/oshi-core/pom.xml

<snip>

[ERROR] Failed to execute goal biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process (default) on project oshi-core: bnd error: null: ConcurrentModificationException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal biz.aQute.bnd:bnd-maven-plugin:5.0.1:bnd-process (default) on project oshi-core: bnd error: null

<snip>

Caused by: org.apache.maven.plugin.MojoExecutionException: bnd error: null
    at aQute.bnd.maven.plugin.AbstractBndMavenPlugin.execute (AbstractBndMavenPlugin.java:451)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

<snip>

Caused by: java.util.ConcurrentModificationException
    at java.util.TreeMap.callMappingFunctionWithCheck (TreeMap.java:742)
    at java.util.TreeMap.computeIfAbsent (TreeMap.java:596)
    at aQute.bnd.osgi.Jar.putResource (Jar.java:337)
    at aQute.bnd.osgi.Jar$2.visitFile (Jar.java:252)
    at aQute.bnd.osgi.Jar$2.visitFile (Jar.java:227)
    at java.nio.file.Files.walkFileTree (Files.java:2804)
    at aQute.bnd.osgi.Jar.buildFromDirectory (Jar.java:226)
    at aQute.bnd.osgi.Jar.<init> (Jar.java:128)
    at aQute.bnd.osgi.Jar.<init> (Jar.java:198)
    at aQute.bnd.osgi.Analyzer.addClasspath (Analyzer.java:2471)
    at aQute.bnd.maven.plugin.AbstractBndMavenPlugin.execute (AbstractBndMavenPlugin.java:190)

<snip>

[dbwiddis]

Based on the TreeMap issues, this may be the breaking change:
https://hg.openjdk.java.net/jdk/jdk/rev/65b345254ca3

[marcphilipp]

We're also seeing that when calling new Jar(jarFile) in JUnit 5:
https://github.com/junit-team/junit5/blob/889088fc7ff8029750a1887e844d9a37d2d5dd08/platform-tooling-support-tests/src/test/java/platform/tooling/support/tests/ManifestTests.java#L45

Here's a stacktrace:
https://ge.junit.org/s/hqmd3sf5tkhtc/tests/5uodjcz5nhszu-w7374ncu5zr7g?open-stacktraces=WzBd

I think the root cause is that Jar#putResource modifies directories in a computeIfAbsent callback here:

bnd/biz.aQute.bndlib/src/aQute/bnd/osgi/Jar.java

Lines 329 to 338 in 298c2b2

	Map<String, Resource> s = directories.computeIfAbsent(getParent(path), dir -> {
	// make ancestor directories
	for (int n; (n = dir.lastIndexOf('/')) > 0;) {
	dir = dir.substring(0, n);
	if (directories.containsKey(dir))
	break;
	directories.put(dir, null);
	}
	return new TreeMap<>();
	});

That is documented to be forbidden in Map:

The default implementation makes no guarantees about detecting if the mapping function modifies this map during computation and, if appropriate, reporting an error. Non-concurrent implementations should override this method and, on a best-effort basis, throw a ConcurrentModificationException if it is detected that the mapping function modifies this map during computation.

[rotty3000]

yeah, we should reverse the algorithm and do a pass adding the parents instead.

[bjhargrave]

Once the build completes, you should be able to test with the 5.1.0-SNAPSHOT build. See https://github.com/bndtools/bnd/blob/master/maven/README.md#using-the-latest-development-snapshot-build-of-the-bnd-maven-plugins.

[dbwiddis]

I can confirm 5.1.0-SNAPSHOT resolves the maven plugin issue. Thanks for the quick turnaround.

[marcphilipp]

@bjhargrave Would it be possible to release this fix in 5.0.2 or is 5.1.0 about to be released soon?

[bjhargrave]

Would it be possible to release this fix in 5.0.2 or is 5.1.0 about to be released soon?

I would like to go with releasing 5.1.0 soon since there are some other commits needed to support Java 15 and build correct bundles. I will discuss with the other maintainers.

[garydgregory]

A release soon would be most welcome as we are seeing this problem blow up Apache Commons CSV builds on Java 15:

• https://pastebin.com/c3H3Zi7X
• https://travis-ci.org/github/apache/commons-csv/jobs/675190349
• https://issues.apache.org/jira/browse/FELIX-6259
  TY!

[dbwiddis]

Looks like only one issue to go which says it'll get done this week...

[bjhargrave]

Yes, we are trying to get to 5.1.0.RC1 by Friday! Busy trying to get the last bits in!

[garydgregory]

@bjhargrave Good luck then! :-)

[garydgregory]

@bjhargrave
Fast-forward one month... any news on 5.1? ;-)

[bjhargrave]

Yes! We have released 5.1.0.RC1. About to kick off the 5.1.0.RC2 build.

[gnodet]

Where are the RC published ? I'd like to get it into the felix maven-bundle-plugin asap.

[pkriens]

https://bndtools.org/installation.html#update-site

[gnodet]

@pkriens strangely, the update-rc points to snapshots, not the rc : https://bndtools.jfrog.io/bndtools/update-rc/biz/aQute/bnd/biz.aQute.bndlib/

[pkriens]

The RCs are snapshots

[gnodet]

Ok, I'll wait for the GA then.

[bjhargrave]

For all interested parties, Bnd 5.1.0 was released yesterday. So you can update to it for Java 15 support.

[momomo]

Fails here too.

[momomo]

I am on oracle btw.

[rotty3000]

Which version of the plugin are you using? This issue was fixed a good long while ago.

…

On Sat., Mar. 6, 2021, 4:03 p.m. momomo, ***@***.***> wrote: I am on oracle btw. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3903 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABD2THNGHB5G222NON5WL3TCKKDXANCNFSM4MFHJA6A> .

[momomo]

Well, the project i was using was using first the old one 2.5.3. So I updated to 4.2.1 because that is what it says here:

https://felix.apache.org/components/bundle-plugin/plugin-info.html

However, I found out that there is an even newer version later, 5.1.1 using search.maven.org.

Now it is fine.

Howeveer, someone ought to update that website!

[bjhargrave]

The link you reference is for Apache Felix and it's maven-bundle-plugin which is not the same as Bnd's bnd-maven-plugin. While they both use Bnd internally, they are distinct plugins from different teams.