Intermittent exception when installing HikariCP instrumentation

[ipimentelmoraes]

Describe the bug

Sometimes, during the agent installation on application startup, it throws an exception when transforming the com.zaxxer.hikari.pool.HikariPool class. When this happens db.client metrics related to our HikariCP connection pools are not reported.

"[otel.javaagent 2023-08-28 13:09:27:283 +0000] [qtp703645946-41] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Failed to handle com.zaxxer.hikari.pool.HikariPool for transformation on class loader jdk.internal.loader.ClassLoaders$AppClassLoader@6a6824be"
"java.lang.IllegalStateException: Field POOL_NORMAL defines an incompatible default value 0"
"\tat net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ValidatingClassVisitor.visitField(TypeWriter.java:2513)"
"\tat org.objectweb.asm.ClassVisitor.visitField(ClassVisitor.java:356)"
"\tat org.objectweb.asm.ClassVisitor.visitField(ClassVisitor.java:356)"
"\tat net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining$WithFullProcessing$RedefinitionClassVisitor.onVisitField(TypeWriter.java:5136)"
"\tat net.bytebuddy.utility.visitor.MetadataAwareClassVisitor.visitField(MetadataAwareClassVisitor.java:278)"
"\tat org.objectweb.asm.ClassReader.readField(ClassReader.java:1137)"
"\tat org.objectweb.asm.ClassReader.accept(ClassReader.java:739)"
"\tat org.objectweb.asm.ClassReader.accept(ClassReader.java:424)"
"\tat net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4014)"
"\tat net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)"
"\tat net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4057)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:12225)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12160)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1800(AgentBuilder.java:11869)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12647)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12579)"
"\tat java.base/java.security.AccessController.doPrivileged(Native Method)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12103)"
"\tat net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)"
"\tat java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)"
"\tat java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)"
"\tat java.base/java.lang.ClassLoader.defineClass1(Native Method)"
"\tat java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)"
"\tat java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)"
"\tat java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)"
"\tat java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)"
"\tat java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)"
"\tat java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)"
"\tat java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)"
"\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)"
"\tat com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)"
"\tat next.jdbc.connection$make_connection.invokeStatic(connection.clj:395)"
"\tat next.jdbc.connection$make_connection.invoke(connection.clj:379)"
"\tat next.jdbc.connection$fn__24491.invokeStatic(connection.clj:418)"
"\tat next.jdbc.connection$fn__24491.invoke(connection.clj:416)"
"\tat next.jdbc.protocols$fn__24291$G__24286__24298.invoke(protocols.clj:24)"
"\tat next.jdbc.result_set$fn__25066.invokeStatic(result_set.clj:919)"
"\tat next.jdbc.result_set$fn__25066.invoke(result_set.clj:842)"
"\tat next.jdbc.protocols$fn__24313$G__24308__24322.invoke(protocols.clj:33)"
"\tat next.jdbc$execute_BANG_.invokeStatic(jdbc.clj:250)"
"\tat next.jdbc$execute_BANG_.invoke(jdbc.clj:237)"
"\tat i80.common.components.postgres.Postgres.execute_BANG_(postgres.clj:157)"
"\tat medusa.db.deadletter$fn__25578$count_by_topic__25583$fn__25584.invoke(deadletter.clj:46)"
"\tat medusa.db.deadletter$fn__25578$count_by_topic__25583.invoke(deadletter.clj:38)"
"\tat medusa.ports.http_in$get_deadletters_metrics.invokeStatic(http_in.clj:64)"
"\tat medusa.ports.http_in$get_deadletters_metrics.invoke(http_in.clj:62)"
"\tat reitit.interceptor$fn__31800$fn__31801.invoke(interceptor.cljc:68)"
"\tat io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:54)"
"\tat io.pedestal.interceptor.chain$try_f.invoke(chain.clj:44)"
"\tat io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic(chain.clj:171)"
"\tat io.pedestal.interceptor.chain$process_all_with_binding.invoke(chain.clj:146)"
"\tat io.pedestal.interceptor.chain$process_all$fn__29609.invoke(chain.clj:188)"
"\tat clojure.lang.AFn.applyToHelper(AFn.java:152)"
"\tat clojure.lang.AFn.applyTo(AFn.java:144)"
"\tat clojure.core$apply.invokeStatic(core.clj:667)"
"\tat clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)"
"\tat clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)"
"\tat clojure.lang.RestFn.invoke(RestFn.java:425)"
"\tat io.pedestal.interceptor.chain$process_all.invokeStatic(chain.clj:186)"
"\tat io.pedestal.interceptor.chain$process_all.invoke(chain.clj:182)"
"\tat io.pedestal.interceptor.chain$enter_all.invokeStatic(chain.clj:235)"
"\tat io.pedestal.interceptor.chain$enter_all.invoke(chain.clj:229)"
"\tat io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:379)"
"\tat io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)"
"\tat io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:389)"
"\tat io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)"
"\tat io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__33685.invoke(servlet_interceptor.clj:351)"
"\tat io.pedestal.http.servlet.FnServlet.service(servlet.clj:28)"
"\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)"
"\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)"
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)"
"\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)"
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)"
"\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)"
"\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)"
"\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)"
"\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)"
"\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)"
"\tat org.eclipse.jetty.server.Server.handle(Server.java:516)"
"\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)"
"\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)"
"\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)"
"\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)"
"\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)"
"\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)"
"\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)"
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)"
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)"
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)"
"\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)"
"\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)"
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)"
"\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)"
"\tat java.base/java.lang.Thread.run(Thread.java:829)"

Steps to reproduce

The exception is intermittent and seemingly random. With the same configuration, most of the times the agent is able to report everything just fine. But other times the exception occurs and the application replica doesn't report any db.client metrics. We noticed the problem when querying some db metrics and realizing some instances were not reporting anything, even when db connections were happening.

To understand the problem better, we activated debug logs and ran a routine to recreate some of our pods from time to time. Then we noticed the correlation between the exception and the metrics not being reported.

Expected behavior

The db.client metrics for HikariCP should always be reported when there are db connections.

Actual behavior

Sometimes, in a seemingly random fashion, the db.client metrics for HikariCP are not being reported even when the application has active db connections.

Javaagent or library instrumentation version

v1.30.0

Environment

JDK: Temurin 11.0.15
OS: Alpine 3.16.1
HikariCP 5.0.1

Additional context

Java Agent properties:

otel.service.name=medusa
otel.metric.export.interval=30000
otel.traces.exporter = none
otel.logs.exporter = none
otel.instrumentation.common.default-enabled=true
# Disabling http.server metrics, as Pedestal causes all the http.route labels to be "/*", turning the metric irrelevant
otel.instrumentation.jetty.enabled=false
otel.instrumentation.servlet.enabled=false

[laurit]

Can you reproduce this at will? I guess we could disable this validation, but the weird thing is that unlike what the exception claims 0 should be a valid value. Looking at https://github.com/raphw/byte-buddy/blob/880027cf43266dcea0feeec5d200188db98baedf/byte-buddy-dep/src/main/java/net/bytebuddy/dynamic/scaffold/TypeWriter.java#L2513 I can't see why or how this check could fail. This field should be an int and 0 should be between min and max value there. If you can reliably reproduce this then perhaps you could set a breakpoint on the line where the exception is thrown and inspect what the min/max/descriptor and other locals are. If you can't reproduce it that reliably then we'll have to come up with something else.
Also the stack trace doesn't look exactly what I'd expect. Could you check at the start of the debug output, that the version printed there is 1.30.0 not something else.

[ipimentelmoraes]

@laurit Unfortunately we don't know how to reproduce the issue reliably. We are thinking about forking the projects and adding more logs specific to this issue whenever possible, so the next time it happens we have more information. Do you have any other suggestions on where we could start or what we should try debugging?

And forgive me about the stack trace, the one I sent was indeed from an older version. Here's one from 1.30.0:

[otel.javaagent 2023-10-22 17:36:02:438 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.30.0
[otel.javaagent 2023-10-22 17:36:02:440 +0000] [main] DEBUG io.opentelemetry.javaagent.tooling.VersionLogger - Running on Java 11.0.15. JVM OpenJDK 64-Bit Server VM - Eclipse Adoptium - 11.0.15+10
...
otel.javaagent 2023-10-22 17:37:47:189 +0000] [qtp745107874-42] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Failed to handle com.zaxxer.hikari.pool.HikariPool for transformation on class loader jdk.internal.loader.ClassLoaders$AppClassLoader@6a6824be
java.lang.IllegalStateException: Field POOL_NORMAL defines an incompatible default value 0
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ValidatingClassVisitor.visitField(TypeWriter.java:2513)
at org.objectweb.asm.ClassVisitor.visitField(ClassVisitor.java:356)
at org.objectweb.asm.ClassVisitor.visitField(ClassVisitor.java:356)
at org.objectweb.asm.ClassVisitor.visitField(ClassVisitor.java:356)
at net.bytebuddy.utility.visitor.MetadataAwareClassVisitor.onVisitField(MetadataAwareClassVisitor.java:293)
at net.bytebuddy.utility.visitor.MetadataAwareClassVisitor.visitField(MetadataAwareClassVisitor.java:278)
at org.objectweb.asm.ClassReader.readField(ClassReader.java:1137)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:739)
at org.objectweb.asm.ClassReader.accept(ClassReader.java:424)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4014)
at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
at net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4062)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:12458)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12393)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1800(AgentBuilder.java:12102)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12880)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12812)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12336)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at next.jdbc.connection$make_connection.invokeStatic(connection.clj:395)
at next.jdbc.connection$make_connection.invoke(connection.clj:379)
at next.jdbc.connection$fn__24491.invokeStatic(connection.clj:418)
at next.jdbc.connection$fn__24491.invoke(connection.clj:416)
at next.jdbc.protocols$fn__24291$G__24286__24298.invoke(protocols.clj:24)
at next.jdbc.result_set$fn__25066.invokeStatic(result_set.clj:919)
at next.jdbc.result_set$fn__25066.invoke(result_set.clj:842)
at next.jdbc.protocols$fn__24313$G__24308__24322.invoke(protocols.clj:33)
at next.jdbc$execute_BANG_.invokeStatic(jdbc.clj:250)
at next.jdbc$execute_BANG_.invoke(jdbc.clj:237)
at i80.common.components.postgres.Postgres.execute_BANG_(postgres.clj:157)
at medusa.db.deadletter$fn__25578$count_by_topic__25583$fn__25584.invoke(deadletter.clj:46)
at medusa.db.deadletter$fn__25578$count_by_topic__25583.invoke(deadletter.clj:38)
at medusa.ports.http_in$get_deadletters_metrics.invokeStatic(http_in.clj:64)
at medusa.ports.http_in$get_deadletters_metrics.invoke(http_in.clj:62)
at reitit.interceptor$fn__31800$fn__31801.invoke(interceptor.cljc:68)
at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:54)
at io.pedestal.interceptor.chain$try_f.invoke(chain.clj:44)
at io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic(chain.clj:171)
at io.pedestal.interceptor.chain$process_all_with_binding.invoke(chain.clj:146)
at io.pedestal.interceptor.chain$process_all$fn__29609.invoke(chain.clj:188)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at io.pedestal.interceptor.chain$process_all.invokeStatic(chain.clj:186)
at io.pedestal.interceptor.chain$process_all.invoke(chain.clj:182)
at io.pedestal.interceptor.chain$enter_all.invokeStatic(chain.clj:235)
at io.pedestal.interceptor.chain$enter_all.invoke(chain.clj:229)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:379)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:389)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:352)
at io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__33685.invoke(servlet_interceptor.clj:351)
at io.pedestal.http.servlet.FnServlet.service(servlet.clj:28)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)

Another curious thing that we noticed now is that we noticed now is that similar errors also occur when transforming other classes, for example:

[otel.javaagent 2023-10-22 17:36:23:374 +0000] [main] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Failed to handle java.util.concurrent.ForkJoinPool for transformation on class loader null
java.lang.IllegalStateException: Field SWIDTH defines an incompatible default value 16
...

and

[otel.javaagent 2023-10-22 17:36:36:641 +0000] [main] DEBUG io.opentelemetry.javaagent.tooling.AgentInstaller$TransformLoggingListener - Failed to handle io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder for transformation on class loader jdk.internal.loader.ClassLoaders$AppClassLoader@6a6824be
java.lang.IllegalStateException: Field DEFAULT_FLOW_CONTROL_WINDOW defines an incompatible default value 1048576

This makes us think that the problem is not related to HikariCP per se, and the lack of metrics is just the sympton. But we are kind of lost in how to investigate further.

[laurit]

How often do you see this? As this exception doesn't make sense I'd probably try with a different java version or disable jit compiler for net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ValidatingClassVisitor.visitField. Also would check whether there are any other instrumentation agents and whether they could somehow modify that method.

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.