Java 17 support #1536

jaroslawr · 2021-09-17T10:15:55Z

The rececently released Java 17 removed Unsafe.defineAnonymousClass(): https://bugs.openjdk.java.net/browse/JDK-8267178

In effect Guice will use ChildClassDefiner where it previously used UnsafeClassDefiner, and will then fail when generating its proxy classes for non-public classes e.g. using @Inject on their constructor, with an error like this:

class XYZ$$EnhancerByGuice$$575108993 cannot access its superclass XYZ (XYZ$$EnhancerByGuice$$575108993 is in unnamed module of loader ChildClassDefiner$ChildLoader @72acb566; XYZ is in unnamed module of loader 'app')

If you trace down where this error comes from in the JDK, the inter-module boundaries are not even checked if the superclass is not public, so there is no chance of fixing this by adding module exports or opens to the codebase that is using Guice. Apparently Hidden Classes are the suggested replacement for Unsafe.defineAnonymousClass:

https://openjdk.java.net/jeps/371

jaroslawr · 2021-09-17T10:19:14Z

(Related: #1529)

mcculls · 2021-09-17T10:43:26Z

Yes, this should be fixed by #1529 (I just need to add some documentation around the new defining process)

GedMarc · 2021-09-17T10:59:29Z

Hey @mcculls
In Java 17 the proxy classes are getting created in an unnamed module instead of the owning class module/ or the com.google.inject module (as with 11->16)

The superclass access check is failing on the proxy class, problem doesn't exist in a lower JDK (except 12, where it popped up and vanished in JDK 13)

Previous behaviour that works

module {
opens my.package.that.injects to com.google.inject;
}

Required changes for JDK 17 (because of superclass access check)

module {
exports my.package.that.inject ; //this is now required because the proxy class is loaded into an unnamed module now)
opens my.package.that.injects to com.google.inject;
}

Exception

Caused by: java.lang.IllegalAccessError: class za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$118282047 (in unnamed module @0x1fdd8364) cannot access class za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit (in module za.co.uwe.assist.v5.web) because module za.co.uwe.assist.v5.web does not export za.co.uwe.assist.v5.web to unnamed module @0x1fdd8364
	at za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$118282047.GUICE$TRAMPOLINE(<generated>)
	at za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$118282047.apply(<generated>)
	at [email protected]_0-SNAPSHOT/com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)

Dropping a JDK version and the issue goes away xD

======================
Full classname legend:
======================
SaveFarmSessionIntoScopeOnCallExit:                              "za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit"
SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$118282047: "za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$118282047"
========================
End of classname legend:
========================

GedMarc · 2021-09-17T11:00:34Z

Forgot to mention - When doing this, the unsafe access exception disappears and this one pops up

mcculls · 2021-09-17T11:36:27Z

@GedMarc can you share an example project or test that recreates the exception?

GedMarc · 2021-09-17T12:02:07Z

Here we are :-
guiced17.zip

profile "raw" for replicating on automatic module names, profile "modular" for executions on pure modules (for jlink/jpackage distributions)

Modular results :-

Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Exception in thread "main" com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) [Guice/ErrorInjectingConstructor]: IllegalAccessError: class Caller$$FastClassByGuice$$1904349 (in unnamed module @0x197d671) cannot access class Caller (in module guiced17) because module guiced17 does not export my.application.to to unnamed module @0x197d671
  at Caller.<init>(Caller.java:10)
  while locating Caller

Learn more:
  https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR

1 error

======================
Full classname legend:
======================
Caller:                            "my.application.to.Caller"
Caller$$FastClassByGuice$$1904349: "my.application.to.Caller$$FastClassByGuice$$1904349"
========================
End of classname legend:
========================

	at [email protected]/com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:251)
	at [email protected]/com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1103)
	at [email protected]/com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1138)
	at guiced17/my.application.to.Caller.main(Caller.java:18)
Caused by: java.lang.IllegalAccessError: class my.application.to.Caller$$FastClassByGuice$$1904349 (in unnamed module @0x197d671) cannot access class my.application.to.Caller (in module guiced17) because module guiced17 does not export my.application.to to unnamed module @0x197d671
	at my.application.to.Caller$$FastClassByGuice$$1904349.GUICE$TRAMPOLINE(<generated>)
	at my.application.to.Caller$$FastClassByGuice$$1904349.apply(<generated>)
	at [email protected]/com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
	at [email protected]/com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
	at [email protected]/com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
	at [email protected]/com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
	at [email protected]/com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1100)
	... 2 more

Raw execution Results

Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Exception in thread "main" com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) [Guice/ErrorInjectingConstructor]: IllegalAccessError: class Caller$$FastClassByGuice$$1660256 (in unnamed module @0x27c86f2d) cannot access class Caller (in module guiced17) because module guiced17 does not export my.application.to to unnamed module @0x27c86f2d
  at Caller.<init>(Caller.java:10)
  while locating Caller

Learn more:
  https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR

1 error

======================
Full classname legend:
======================
Caller:                            "my.application.to.Caller"
Caller$$FastClassByGuice$$1660256: "my.application.to.Caller$$FastClassByGuice$$1660256"
========================
End of classname legend:
========================

	at [email protected]/com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:251)
	at [email protected]/com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1103)
	at [email protected]/com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1138)
	at guiced17/my.application.to.Caller.main(Caller.java:18)
Caused by: java.lang.IllegalAccessError: class my.application.to.Caller$$FastClassByGuice$$1660256 (in unnamed module @0x27c86f2d) cannot access class my.application.to.Caller (in module guiced17) because module guiced17 does not export my.application.to to unnamed module @0x27c86f2d
	at my.application.to.Caller$$FastClassByGuice$$1660256.GUICE$TRAMPOLINE(<generated>)
	at my.application.to.Caller$$FastClassByGuice$$1660256.apply(<generated>)
	at [email protected]/com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
	at [email protected]/com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
	at [email protected]/com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
	at [email protected]/com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
	at [email protected]/com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1100)
	... 2 more

mcculls · 2021-09-17T13:30:03Z

Thanks, so the issue with the module example is that recent JDKs disallow access to sun.misc.Unsafe from modules unless they requires jdk.unsupported. Without this requires the Unsafe class is not available and Guice falls back to using a child classloader. Note if Guice is on the classpath, it can access sun.misc.Unsafe - so this is only an issue when Guice is on the module path either as an automatic module or a pure module.

For the pure module case adding requires jdk.unsupported to the Guice module should be enough.

For the automatic module case you can either add requires jdk.unsupported to the example module, or use --add-modules jdk.unsupported on the command-line. (As long as some module requires jdk.unsupported or it's added via the command-line then Guice running as an automatic module will be allowed to also access it.)

I'll have a think about how best to do this in the build (which will probably involve making it a pure module)

GedMarc · 2021-09-17T14:16:52Z

Hmm I can't confirm that,

I've added the requires clause to both the core as a pure module, and the consumer as an automatic module, the same exception - (Only in 17) - also to command line

Caused by: java.lang.IllegalAccessError: class za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$114333740 (in unnamed module @0x403b2e0b) cannot access class za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit (in module za.co.uwe.assist.v5.web) because module za.co.uwe.assist.v5.web does not export za.co.uwe.assist.v5.web to unnamed module @0x403b2e0b
	at za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$114333740.GUICE$TRAMPOLINE(<generated>)
	at za.co.uwe.assist.v5.web.SaveFarmSessionIntoScopeOnCallExit$$FastClassByGuice$$114333740.apply(<generated>)
	at [email protected]_0-SNAPSHOT/com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
	at [email protected]_0-SNAPSHOT/com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)

Removing the added exports and adding in the requires still ends with the same result

I'm attaching the jar(zip) with the module-info updated to include jdk.unsupported,
Can use the previous replicator with jdk.unsupported as well to produce the same exception

guice-0.0.0_0-SNAPSHOT.zip

GedMarc · 2021-09-17T14:19:25Z

The module shade/build is configured here (maybe to look at to assist not sure) - https://github.com/GedMarc/GuicedEE-Services/tree/master/guice-core

mcculls · 2021-09-17T14:37:19Z

@GedMarc have you pulled in the changes from #1529 ?

GedMarc · 2021-09-17T14:46:28Z

i haven't sorry! let me do that quick

GedMarc · 2021-09-18T11:29:51Z

@mcculls Purring like a kitten sir!

the requires clause only necessary on the guice module, otherwise everything looks good -
checked (obviously all the default as well)

aop & private modules
generic types binding
persistence modules
servlet modules

Happy panda :) You're the man!

GedMarc · 2021-09-18T11:33:11Z

putting the modular jar here with my module-info just incase a multi release jar becomes a thing (doesn't include error prone annotations)

guice.zip

module-info.zip

wendigo · 2021-10-05T08:01:15Z

@mcculls is there any plan to make a release that includes recent JDK17 related improvements in the near future?

vjh0107 · 2021-10-24T20:56:10Z

@GedMarc is there a way to use just Gradles without module-info?

GedMarc · 2021-10-25T06:33:52Z

@vjh0107 I'm not entirely sure why you would use a java version above 8 without a module-info, that's quite a performance degradation running a modular VM in classpath mode - you maybe want to think about staying on 8 if that is the path you are thinking of taking - Essentially though, that isn't a viable path so I'm not sure I'm actually going to try

You can take the guice artifact and install it into your local repository, removing the module-info file, you can also create a shade that excludes the module-info file from the artifact,
Otherwise clone the repository on this commit and perform the build and deploy locally :)

norrisjeremy · 2021-10-25T12:01:14Z

Hi @GedMarc,

I'm curious about this performance degradation you mentioned: do you have more details as to how using a modularized jar on the classpath degrades performance?

Thanks,
Jeremy

GedMarc · 2021-10-25T12:24:03Z

@norrisjeremy AMD is now a standard across all languages and has filtered down to Java, so aside from all the obvious reasons on why that is and all those benefits - I'm going to try list everything off the top of my head and mention what it replaces in java for the same result, just native.

ClassPath access from JDK 9 and up is now a backwards compatible strategy being deprecated rather than a first-class implementation. These changes include not using the new Fast ClassLoader, not using the new method definition structure, not using the JDK14 class structure (goodbye 5 abstract class limit), as these features are present only in native runtimes, or completely strict-modular defined applications. The module enhanced ClassLoader for fine-tuned memory management and exposures (replacing oSGI), the ServiceLoader mechanisms derived from proper AMD definitions, the JRT virtual filesystem that replaces java virtual file systems completely (wildfly vfs, etc etc etc) - my lawd the performance, and the new file system management for class identification and share-classes between JDK's
In consideration of the new features and capabilities of a modular virtual machine the following technologies are no longer required and you get all benefits + more from using raw Java SE

Java* EE (Obsolete) / Servlets and HTTP access is now located in java.net module, persistence is still required to a degree, the rest like WS and REST are all flavours of HTTP servers and calls and can be configured (some good examples of frameworkless json servers) using the stock tools
oSGI
virtual file systems
Contextual Dependency Injection (CDI) java.enterprise package / downward injections are now performed through the new implementation of ServiceLoader. Illegal Injecting is now not allowed (at all from JDK 16), DI is now the way to go.
This is a strong opinion but --- Everything that was thrown away when Oracle did it, was justified, if you actually used it... (nobody kill me on that)

So good news, you're on Guice which is already a modular DI, is not externally configured, and defines injections properly - was just way ahead of its time. Just got to remember to deploy JRE's, and build them accordingly - JDK in production, especially now, still frowned upon

I've found no less than a 60% boot up performance time in native runtimes, modules provide exceedingly great memory management to anything else, and not needing to bundle an entire damn framework to do microservicing - well, also it's AMD so all of those reasons why as well.

Opinionated, but proven. No one yet has done an article on the performance differences between enterprise applications in native runtimes vs classpath management.

norrisjeremy · 2021-10-25T13:06:50Z

Hi @GedMarc,

I'm still not sure I fully understand your earlier statement discouraging usage of Java9+ with the classpath instead of modulepath: I don't believe I can recall having seen any direct statements from Oracle officially deprecating the usage of the classpath in-lieu of the modulepath, but perhaps I have missed something?

Thanks,
Jeremy

vjh0107 · 2021-10-25T20:07:10Z

@GedMarc
The reason why I asked that question, I can't set module-info in my project. However, Class CastException seems to occur because I didn't set up the module-info when I used Java 16. I'm currently facing this problem. I'd appreciate it if you could answer if you knew.

https://stackoverflow.com/questions/69699624/guice-unnamed-module-error-java-lang-classcastexception-cannot-be-cast-to-cla

thank you

jhaber · 2021-11-08T22:12:19Z

Would it be possible to publish a new Guice release that includes #1529? I wasn't able to get our services to start up on Java 17 without those changes

anthraxx · 2021-12-10T16:57:27Z

Can we pretty please get a new release out? 🐈

FreeGuwop · 2021-12-30T18:27:38Z

Can we put out a new release of Guice?

anthraxx · 2022-01-20T15:57:04Z

@markmarch @ad-fu @java-team-github-bot Sorry for dropping yet another comment, but I hope the explicit mentions here bring in some progress.
There are life signs as commits in the tree, but the lack of a release is really a hard blocker and very painful currently. Can we pretty please get a new release so we can make use of the fixes? 😿

markmarch · 2022-01-20T19:21:48Z

We are working on a new release but there are currently some test failures when running with Java 17.

@mcculls If you have time, can you look into some of the test failures since you are most familiar with the bytecode gen stuff and those tests failures all seem to be related to that?

With bazel installed, you can reproduce the issue with the following command from the root of the repo:

bazel test --java_runtime_version=remotejdk_17 //...

I'll also spend some time looking into this.

The original plan is to migrate from maven to bazel then create a new release that also include some of the kotlin work we've done that hasn't been released but I probably will just cut a new release once the test failures are fixed and punt the bazel and kotlin work to next release.

mcculls · 2022-01-20T20:19:23Z

Hi @markmarch some of those failing tests rely on parsing stack traces to verify what mechanism was used to call the method. Java17 hides certain types of frames by default, particularly frames relating to anonymous nest-mates, so you need to add -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames so those tests can find the frames they're looking for:

https://github.com/google/guice/blob/master/core/pom.xml#L104

Note this is just for test verification purposes - in everyday use you wouldn't worry about not seeing these frames in stack traces.

EDIT: also the assisted-inject failure in SubpackageTest.testReflectionFallbackWorks is expected when running that test with Java17 because that particular fallback only works on Java16 or below. This isn't an issue in practice because the lookup approach which is attempted first will pass on Java17, so that fallback is never needed (it's only really needed on Java8.)

SubpackageTest.testReflectionFallbackWorks just needs an assumeTrue so it doesn't run on Java17 or later.

EDIT: lastly, the JPA failures look related to the version of Mockito used, upgrading Mockito should fix those.

…1536 (comment) - skip reflection fallback test that only works for java version < 17 - add required jvm flags to mirror maven setup - upgrade mockito version PiperOrigin-RevId: 423168879

…1536 (comment) - skip reflection fallback test that only works for java version < 17 - add required jvm flags to mirror maven setup - upgrade mockito version PiperOrigin-RevId: 423389524

…ase. closes #1536 PiperOrigin-RevId: 423924393

copybara-service bot mentioned this issue Jan 21, 2022

Fix tests that break with Java 17 base on suggestions from @mcculls https://github.com/google/guice/issues/1536#issuecomment-1017894441 #1567

Merged

copybara-service bot pushed a commit that referenced this issue Jan 25, 2022

bump snapshot version, update README & prior API xml after 5.1.0 rele…

b781a73

…ase. closes #1536 PiperOrigin-RevId: 423924393

copybara-service bot mentioned this issue Jan 25, 2022

bump snapshot version, update README & prior API xml after 5.1.0 release. #1572

Closed

copybara-service bot closed this as completed in 6925c5c Jan 25, 2022

thinker0 mentioned this issue Mar 11, 2022

guice-4.0 does not support jdk17 apache/incubator-heron#3790

Closed

cacoco mentioned this issue Mar 23, 2023

[Scala] Use sbt official docker images the-benchmarker/web-frameworks#6159

Merged

cxzl25 mentioned this issue Nov 22, 2023

[CELEBORN-856] Add mapreduce integration test apache/celeborn#2073

Closed

pan3793 mentioned this issue Jul 1, 2024

HADOOP-19216. Upgrade Guice from 4.0 to 5.1.0 to support Java 17 apache/hadoop#6913

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java 17 support #1536

Java 17 support #1536

jaroslawr commented Sep 17, 2021

jaroslawr commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 18, 2021

GedMarc commented Sep 18, 2021

wendigo commented Oct 5, 2021

vjh0107 commented Oct 24, 2021

GedMarc commented Oct 25, 2021

norrisjeremy commented Oct 25, 2021

GedMarc commented Oct 25, 2021

norrisjeremy commented Oct 25, 2021

vjh0107 commented Oct 25, 2021

jhaber commented Nov 8, 2021

anthraxx commented Dec 10, 2021

FreeGuwop commented Dec 30, 2021

anthraxx commented Jan 20, 2022

markmarch commented Jan 20, 2022

mcculls commented Jan 20, 2022 •

edited

Loading

Java 17 support #1536

Java 17 support #1536

Comments

jaroslawr commented Sep 17, 2021

jaroslawr commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 17, 2021

mcculls commented Sep 17, 2021

GedMarc commented Sep 17, 2021

GedMarc commented Sep 18, 2021

GedMarc commented Sep 18, 2021

wendigo commented Oct 5, 2021

vjh0107 commented Oct 24, 2021

GedMarc commented Oct 25, 2021

norrisjeremy commented Oct 25, 2021

GedMarc commented Oct 25, 2021

norrisjeremy commented Oct 25, 2021

vjh0107 commented Oct 25, 2021

jhaber commented Nov 8, 2021

anthraxx commented Dec 10, 2021

FreeGuwop commented Dec 30, 2021

anthraxx commented Jan 20, 2022

markmarch commented Jan 20, 2022

mcculls commented Jan 20, 2022 • edited Loading

mcculls commented Jan 20, 2022 •

edited

Loading