Tests which use java stub template with large classpaths fails because of local_jdk implicit dependency

[ittaiz]

Description of the problem / feature request:

Running a java_test (we're actually using scala_junit_test but that uses the java_stub_template.txt) with a very long classpath fails since the template assumes existence of local_jdk/bin/jar in runfiles.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I'll try to work on a full repro but given a target with a very long classpath using the java_stub_template (java_test/ scala_junit_test) try to test it with bazel 0.19.0 (maybe reproduces earlier) and it will fail with .../foo.runfiles/local_jdk/bin/jar: No such file or directory

I've worked around it for now by adding the following data dependency explicitly: "@bazel_tools//tools/jdk:jar",.
This generates a warning about using a deprecated dependency.

What operating system are you running Bazel on?

OS X

What's the output of bazel info release?

release 0.19.0

Have you found anything relevant by searching the web?

java_stub_template relevant line:

bazel/src/main/java/com/google/devtools/build/lib/bazel/rules/java/java_stub_template.txt

Line 354 in 1afef64

create_and_run_classpath_jar "local_jdk/bin/jar"

I think this is related to @cushon's work on #5594

[ittaiz]

cc @meisterT

[meisterT]

cc @iirina

[cushon]

@laszlocsomor @meteorcloudy

I thought we were only using the classpath jar hack on windows, but that changed in 102ce6d because not all JDKs support @argument files. Note that OpenJDK 9 and newer do support parameter files, so there's a more principled fix available in that case: #6354

[ittaiz]

For now we still use JDK 8... I think we'll move in a few months but now we have a problem. Also my workaround won't work on RBE, right? Is there a workaround I can do to use current java runtime? Note I'm willing to mangle the java_stub_template in my custom test rule for the time being until bazel releases a fix (I guess that will probably land only at February)

…

On Tue, Dec 18, 2018 at 7:21 PM Liam Miller-Cushon ***@***.***> wrote: @laszlocsomor <https://github.com/laszlocsomor> @meteorcloudy <https://github.com/meteorcloudy> I thought we were only using the classpath jar hack on windows, but that changed in 102ce6d <102ce6d> because not all JDKs support @argument files. Note that OpenJDK 9 and newer *do* support parameter files, so there's a more principled fix available in that case: #6354 <#6354> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6955 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIFyFbEYiBWZ8ZMQ7_QhhJNZHBF2Dqks5u6SQFgaJpZM4ZXsIw> .

[ittaiz]

any thoughts? also #6354 is still open

[laszlocsomor]

I'm not a domain expert, but I think Bazel could set JARBIN akin to JAVABIN as it does here:

bazel/src/main/java/com/google/devtools/build/lib/bazel/rules/java/java_stub_template.txt

Lines 231 to 232 in f55ae7a

	# Set JAVABIN to the path to the JVM launcher.
	%javabin%

bazel/src/main/java/com/google/devtools/build/lib/bazel/rules/java/BazelJavaSemantics.java

Lines 298 to 300 in f55ae7a

	Substitution.of(
	"%javabin%",
	JavaCommon.getJavaBinSubstitutionFromJavaExecutable(ruleContext, javaExecutable)));

[beala-stripe]

I have also run into this issue in a java_binary rule while setting the --javabase and --host_javabase to a custom target (an Azul JDK8). Thankfully @ittaiz's workaround worked (adding a data dep to the target: data = ["@local_jdk//:bin/jar"])

[ittaiz]

Any news about this?

[beala-stripe]

I'm also eager for a fix for this. Although adding the data dep gets the build working, I worry that depending on the local jdk negatively affects caching.

[ittaiz]

@cushon @laszlocsomor's suggestion sounds really reasonable and small. WDYT?
An even simpler (but hackier) solution can be to get the parent directory of JAVABIN and then append jar to it.

[ittaiz]

@cushon @lberki I've implemented the above in rules_scala (copied the stub template file).
WDYT? We need this in many more places and don't want to use the deprecated local_jdk.

[ittaiz]

@cushon @lberki wdyt?

[ittaiz]

Any news?

[joshrosen-stripe]

Maybe we should submit a PR to more clearly show the effective diff? As it stands now, this discussion links to PRs against bazelbuild/rules_scala and in that PR it's a little tricky to see the actual net change because it duplicates the entire java stub template. If we submitted a PR directly against bazelbuild/bazel then it might be easier to reason about exactly what's being proposed.

[iirina]

@ittaiz is this still happening?

[TheImplementer]

We've bumped into this issue as well, it looks like it's still happening. (bazel v2.2.0)

[ittaiz]

I think (not 100% sure) that @katre's fix solved the issue.

#11179

[hvadehra]

Closing based on recent comments, please re-open if the issue persists.