[BUG]Dead lock with BadRequestException.printStackTrace() #19657
Comments
ghost
added
needs-triage
|
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @kushagraThapar, @anfeldma-ms |
joshfree
added
Client
ghost
removed
the
needs-triage
|
Thanks for reporting this issue on azure-cosmos-4.12.0, @Horcrux7. @kushagraThapar could you please follow up? @anuchandy can assist from the azure sdk team as needed. |
joshfree
added
bug
|
@Horcrux7 - thanks for reporting this issue, I will investigate it. Is there a way to reproduce this issue ? |
|
The problem seems very time critical. The full code was not many more. Can be that you need to set break points or sleeps to produce it. The exception occur because the POJO object has not a field with |
|
@Horcrux7 - can you please mention the JDK version being used ? |
I had this with different JDK versions. The order of the synchronized in your code is independent of the JDK. The java version with which the thread dump was created can you see the dump:
I does not understand the internals of the cosmos driver. If this prevent the large count of IllegalAccessErrors with afterburner this will be nice. But I think the module-info has no effect because I does not use Java modules, I use only jar files with a classpath. |
xinlian12
added
the
cosmos:v4-item
ghost
added
the
needs-team-attention
|
Closing because of inactivity. @Horcrux7 please re-open if the issue still persists, thanks! |
Describe the bug
A dead lock with BadRequestException.printStackTrace(). An exception should contains only static information that not need any synchronized. The method can be called on any time and any thread.
A reverse synchronized order is the next fail.
Exception or Stack Trace
2021-03-05 12:22:13
Full thread dump Java HotSpot(TM) 64-Bit Server VM (11.0.7+3-LTS mixed mode):
Threads class SMR info:
_java_thread_list=0x00000179c4ede070, length=65, elements={
0x000001799c1ea000, 0x00000179bce59000, 0x00000179bce62000, 0x00000179bd7f2800,
0x00000179bce70000, 0x00000179bce70800, 0x00000179bd7f8000, 0x00000179bd7fe000,
0x00000179bcdef800, 0x00000179bdc76000, 0x00000179bdbae000, 0x00000179bdbe0800,
0x00000179be238000, 0x00000179be83d800, 0x00000179bed20000, 0x00000179be9df000,
0x00000179beb14800, 0x00000179bf2f0000, 0x00000179bf1e9000, 0x00000179c3fc0800,
0x00000179c3fc1000, 0x00000179c3fc2800, 0x00000179c3fc3800, 0x00000179c3fc4800,
0x00000179c3fc7000, 0x00000179c3fc5000, 0x00000179bf684000, 0x00000179bf686800,
0x00000179bf689000, 0x00000179bf685000, 0x00000179bf687800, 0x00000179bf688000,
0x00000179bf685800, 0x00000179bf683800, 0x00000179bf68e000, 0x00000179bf68e800,
0x00000179bf68f800, 0x00000179bf68a000, 0x00000179bf68a800, 0x00000179bf690800,
0x00000179bf691000, 0x00000179bf68b800, 0x00000179bf692000, 0x00000179bf68c000,
0x00000179bf68d000, 0x00000179c4e58000, 0x00000179c4e5c000, 0x00000179c4e59000,
0x00000179c4e5d000, 0x00000179c4e5b800, 0x00000179c4e57800, 0x00000179c4e5e000,
0x00000179c4e5e800, 0x00000179c4e59800, 0x00000179c4e66000, 0x00000179c4e62000,
0x00000179c4e60800, 0x00000179c4e61000, 0x00000179c4e65000, 0x00000179c4e62800,
0x00000179c4e5a800, 0x00000179c4e63800, 0x00000179c4e64800, 0x00000179c4ca2000,
0x00000179c4ca0800
}
"main" #1 prio=5 os_prio=0 cpu=2609.38ms elapsed=549.32s tid=0x000001799c1ea000 nid=0x3aa0 waiting for monitor entry [0x0000006c58dfe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.logging.StreamHandler.publish([email protected]/StreamHandler.java:194)
- waiting to lock <0x0000000084b3a860> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish([email protected]/ConsoleHandler.java:95)
at java.util.logging.Logger.log([email protected]/Logger.java:979)
at java.util.logging.Logger.doLog([email protected]/Logger.java:1006)
at java.util.logging.Logger.log([email protected]/Logger.java:1117)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._reportProblem(OptimizedBeanPropertyWriter.java:131)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._handleProblem(OptimizedBeanPropertyWriter.java:111)
- locked <0x0000000084b3aab8> (a com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter)
at com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter.serializeAsField(ObjectFieldPropertyWriter.java:47)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1838)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:291)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:272)
at com.fasterxml.jackson.databind.SerializerProvider.defaultSerializeValue(SerializerProvider.java:1119)
at com.fasterxml.jackson.databind.node.POJONode.serialize(POJONode.java:115)
at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:39)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:20)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4409)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3663)
at com.azure.cosmos.CosmosException.toString(CosmosException.java:342)
at java.lang.String.valueOf([email protected]/String.java:2951)
at java.io.PrintStream.println([email protected]/PrintStream.java:897)
at java.lang.Throwable$WrappedPrintStream.println([email protected]/Throwable.java:752)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:659)
- locked <0x0000000081bee978> (a java.io.PrintStream)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:648)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:639)
at TestCosmos.main(TestCosmos.java:26)
Locked ownable synchronizers:
- None
"Reference Handler" #2 daemon prio=10 os_prio=2 cpu=0.00ms elapsed=549.29s tid=0x00000179bce59000 nid=0x255c waiting on condition [0x0000006c594fe000]
java.lang.Thread.State: RUNNABLE
at java.lang.ref.Reference.waitForReferencePendingList([email protected]/Native Method)
at java.lang.ref.Reference.processPendingReferences([email protected]/Reference.java:241)
at java.lang.ref.Reference$ReferenceHandler.run([email protected]/Reference.java:213)
Locked ownable synchronizers:
- None
"Finalizer" #3 daemon prio=8 os_prio=1 cpu=0.00ms elapsed=549.29s tid=0x00000179bce62000 nid=0x1804 in Object.wait() [0x0000006c595fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x00000000819a1960> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
- waiting to re-lock in wait() <0x00000000819a1960> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
at java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)
Locked ownable synchronizers:
- None
"Signal Dispatcher" #4 daemon prio=9 os_prio=2 cpu=0.00ms elapsed=549.28s tid=0x00000179bd7f2800 nid=0x7f4 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Attach Listener" #5 daemon prio=5 os_prio=2 cpu=78.13ms elapsed=549.28s tid=0x00000179bce70000 nid=0x1fb4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"C2 CompilerThread0" #6 daemon prio=9 os_prio=2 cpu=7703.13ms elapsed=549.28s tid=0x00000179bce70800 nid=0x3ad0 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
No compile task
Locked ownable synchronizers:
- None
"C1 CompilerThread0" #9 daemon prio=9 os_prio=2 cpu=2468.75ms elapsed=549.28s tid=0x00000179bd7f8000 nid=0x2f40 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
No compile task
Locked ownable synchronizers:
- None
"Sweeper thread" #10 daemon prio=9 os_prio=2 cpu=109.38ms elapsed=549.28s tid=0x00000179bd7fe000 nid=0x3a34 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Common-Cleaner" #11 daemon prio=8 os_prio=1 cpu=0.00ms elapsed=549.20s tid=0x00000179bcdef800 nid=0xce4 in Object.wait() [0x0000006c59bff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x00000000819a24f0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
- waiting to re-lock in wait() <0x00000000819a24f0> (a java.lang.ref.ReferenceQueue$Lock)
at jdk.internal.ref.CleanerImpl.run([email protected]/CleanerImpl.java:148)
at java.lang.Thread.run([email protected]/Thread.java:834)
at jdk.internal.misc.InnocuousThread.run([email protected]/InnocuousThread.java:134)
Locked ownable synchronizers:
- None
"JDWP Transport Listener: dt_socket" #12 daemon prio=10 os_prio=0 cpu=125.00ms elapsed=549.16s tid=0x00000179bdc76000 nid=0xeec runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"JDWP Event Helper Thread" #13 daemon prio=10 os_prio=0 cpu=15.63ms elapsed=549.16s tid=0x00000179bdbae000 nid=0x3578 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"JDWP Command Reader" #14 daemon prio=10 os_prio=0 cpu=46.88ms elapsed=549.16s tid=0x00000179bdbe0800 nid=0x2fe8 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Service Thread" #15 daemon prio=9 os_prio=0 cpu=0.00ms elapsed=548.67s tid=0x00000179be238000 nid=0x3ba8 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Default Timer" #16 daemon prio=5 os_prio=0 cpu=218.75ms elapsed=548.20s tid=0x00000179be83d800 nid=0x39ac waiting for monitor entry [0x0000006c5a4fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:659)
- waiting to lock <0x0000000081bee978> (a java.io.PrintStream)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:648)
at com.inet.persistence.Persistence$Singleton.createInstance(Persistence.java:402)
at com.inet.persistence.Persistence$Singleton.getInstance(Persistence.java:305)
- locked <0x0000000081e177e8> (a java.lang.Class for com.inet.persistence.Persistence$Singleton)
at com.inet.persistence.Persistence.getInstance(Persistence.java:47)
at com.inet.persistence.spi.events.NodeCounter$1.runImpl(NodeCounter.java:37)
at com.inet.thread.timer.DefaultTimerTask.run(DefaultTimerTask.java:22)
at java.util.TimerThread.mainLoop([email protected]/Timer.java:556)
at java.util.TimerThread.run([email protected]/Timer.java:506)
Locked ownable synchronizers:
- None
"ForkJoinPool.commonPool-worker-5" #19 daemon prio=5 os_prio=0 cpu=46.88ms elapsed=548.06s tid=0x00000179bed20000 nid=0x1740 waiting for monitor entry [0x0000006c5a7fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.io.PrintStream.write([email protected]/PrintStream.java:603)
- waiting to lock <0x0000000081bee978> (a java.io.PrintStream)
at java.io.PrintStream.print([email protected]/PrintStream.java:745)
at java.lang.ThreadGroup.uncaughtException([email protected]/ThreadGroup.java:1057)
at java.lang.ThreadGroup.uncaughtException([email protected]/ThreadGroup.java:1050)
at java.lang.Thread.dispatchUncaughtException([email protected]/Thread.java:2001)
Locked ownable synchronizers:
- None
"Logging Spooler" #20 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=548.06s tid=0x00000179be9df000 nid=0x3a20 in Object.wait() [0x0000006c5a8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x0000000081c88210> (a [Lcom.inet.logging.LoggingEvent;)
at java.lang.Object.wait([email protected]/Object.java:328)
at com.inet.logging.LogBuffer.next(LogBuffer.java:54)
- waiting to re-lock in wait() <0x0000000081c88210> (a [Lcom.inet.logging.LoggingEvent;)
at com.inet.logging.LogManager$1.run(LogManager.java:71)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"Configuration change monitor" #22 daemon prio=5 os_prio=0 cpu=31.25ms elapsed=547.99s tid=0x00000179beb14800 nid=0x1278 waiting for monitor entry [0x0000006c5aafd000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.inet.persistence.Persistence$Singleton.getInstance(Persistence.java:304)
- waiting to lock <0x0000000081e177e8> (a java.lang.Class for com.inet.persistence.Persistence$Singleton)
at com.inet.persistence.Persistence.getInstance(Persistence.java:47)
at com.inet.config.ConfigurationManagerImpl$ConfigurationMonitor.lambda$0(ConfigurationManagerImpl.java:180)
at com.inet.config.ConfigurationManagerImpl$ConfigurationMonitor$$Lambda$76/0x0000000100142040.run(Unknown Source)
at java.security.AccessController.doPrivileged([email protected]/Native Method)
at com.inet.config.ConfigurationManagerImpl$ConfigurationMonitor.run(ConfigurationManagerImpl.java:165)
Locked ownable synchronizers:
- None
"Heap Cache Writer 1" #24 daemon prio=10 os_prio=2 cpu=0.00ms elapsed=547.94s tid=0x00000179bf2f0000 nid=0xda0 in Object.wait() [0x0000006c5abfd000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x0000000081c89318> (a com.inet.cache.internal.MemoryObserver)
at com.inet.cache.internal.MemoryObserver$CacheWriter.lambda$0(MemoryObserver.java:441)
- waiting to re-lock in wait() <0x0000000081c89318> (a com.inet.cache.internal.MemoryObserver)
at com.inet.cache.internal.MemoryObserver$CacheWriter$$Lambda$85/0x000000010017a040.run(Unknown Source)
at java.security.AccessController.doPrivileged([email protected]/Native Method)
at com.inet.cache.internal.MemoryObserver$CacheWriter.run(MemoryObserver.java:417)
Locked ownable synchronizers:
- None
"Heap Cache Writer 2" #25 daemon prio=10 os_prio=2 cpu=15.63ms elapsed=547.94s tid=0x00000179bf1e9000 nid=0x100c in Object.wait() [0x0000006c5acfd000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x0000000081c89318> (a com.inet.cache.internal.MemoryObserver)
at com.inet.cache.internal.MemoryObserver$CacheWriter.lambda$0(MemoryObserver.java:441)
- waiting to re-lock in wait() <0x0000000081c89318> (a com.inet.cache.internal.MemoryObserver)
at com.inet.cache.internal.MemoryObserver$CacheWriter$$Lambda$85/0x000000010017a040.run(Unknown Source)
at java.security.AccessController.doPrivileged([email protected]/Native Method)
at com.inet.cache.internal.MemoryObserver$CacheWriter.run(MemoryObserver.java:417)
Locked ownable synchronizers:
- None
"Thread-1" #27 daemon prio=5 os_prio=0 cpu=500.00ms elapsed=546.48s tid=0x00000179c3fc0800 nid=0x18d8 waiting on condition [0x0000006c5a5ff000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821a7790> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"pool-1-thread-1" #28 prio=5 os_prio=0 cpu=218.75ms elapsed=546.28s tid=0x00000179c3fc1000 nid=0x30fc waiting on condition [0x0000006c5b0fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821a7ae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-1" #29 daemon prio=5 os_prio=0 cpu=609.38ms elapsed=546.06s tid=0x00000179c3fc2800 nid=0xa2c waiting on condition [0x0000006c5b1fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821ab690> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-1" #30 daemon prio=5 os_prio=0 cpu=296.88ms elapsed=545.75s tid=0x00000179c3fc3800 nid=0xc30 runnable [0x0000006c5b2fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821aae30> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821aada8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-2" #31 daemon prio=5 os_prio=0 cpu=15.63ms elapsed=545.75s tid=0x00000179c3fc4800 nid=0x2a20 runnable [0x0000006c5b3fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821db038> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821dc060> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-3" #32 daemon prio=5 os_prio=0 cpu=640.63ms elapsed=545.75s tid=0x00000179c3fc7000 nid=0x1168 runnable [0x0000006c5b4fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821eb530> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821bfef0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-4" #33 daemon prio=5 os_prio=0 cpu=31.25ms elapsed=545.75s tid=0x00000179c3fc5000 nid=0x1558 runnable [0x0000006c5b5fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821d5db8> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821d5d30> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-5" #34 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=545.75s tid=0x00000179bf684000 nid=0x3998 runnable [0x0000006c5b6fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821cec48> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821c8c20> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-6" #35 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=545.75s tid=0x00000179bf686800 nid=0x3838 runnable [0x0000006c5b7fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821c3938> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821c3808> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-7" #36 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=545.75s tid=0x00000179bf689000 nid=0x6f4 runnable [0x0000006c5b8fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821b8478> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821b8348> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"reactor-http-nio-8" #37 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=545.75s tid=0x00000179bf685000 nid=0x2f3c runnable [0x0000006c5b9fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x00000000821ad128> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000000821acfe8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-2" #38 daemon prio=5 os_prio=0 cpu=93.75ms elapsed=544.99s tid=0x00000179bf687800 nid=0x21cc waiting on condition [0x0000006c5bcff000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821d5eb8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-monitoring-rntbd-endpoints[1]" #40 daemon prio=1 os_prio=-2 cpu=0.00ms elapsed=544.91s tid=0x00000179bf688000 nid=0xcb8 waiting on condition [0x0000006c5bdff000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084bafb00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.LinkedBlockingQueue.poll([email protected]/LinkedBlockingQueue.java:458)
at io.netty.util.concurrent.SingleThreadEventExecutor.takeTask(SingleThreadEventExecutor.java:256)
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:64)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"Thread-10" #41 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.88s tid=0x00000179bf685800 nid=0x17a0 waiting on condition [0x0000006c5befe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084b3b3a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-3" #42 daemon prio=5 os_prio=0 cpu=125.00ms elapsed=544.88s tid=0x00000179bf683800 nid=0x3bb8 waiting on condition [0x0000006c5bffe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e11a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-channel-acquisition-timer[1]" #43 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.66s tid=0x00000179bf68e000 nid=0x20f4 waiting on condition [0x0000006c5c0ff000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep([email protected]/Native Method)
at io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-pending-acquisition-expirator[1]" #44 daemon prio=5 os_prio=0 cpu=15.63ms elapsed=544.66s tid=0x00000179bf68e800 nid=0x17f4 waiting on condition [0x0000006c5c1fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084bafe48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.LinkedBlockingQueue.poll([email protected]/LinkedBlockingQueue.java:458)
at io.netty.util.concurrent.SingleThreadEventExecutor.takeTask(SingleThreadEventExecutor.java:256)
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:64)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-1" #45 daemon prio=5 os_prio=0 cpu=3390.63ms elapsed=544.61s tid=0x00000179bf68f800 nid=0x204c runnable [0x0000006c5c2fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084ab2a78> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084ab28c8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:814)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-2" #46 daemon prio=5 os_prio=0 cpu=3796.88ms elapsed=544.61s tid=0x00000179bf68a000 nid=0x3b6c runnable [0x0000006c5c3fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084bb08c8> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084bb0718> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:814)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-3" #47 daemon prio=5 os_prio=0 cpu=3609.38ms elapsed=544.45s tid=0x00000179bf68a800 nid=0x37b8 runnable [0x0000006c5bbfe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084a767b0> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084a76600> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:814)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-4" #48 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.37s tid=0x00000179bf690800 nid=0x270c runnable [0x0000006c5c4fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084abea78> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084abe9f0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-5" #49 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.38s tid=0x00000179bf691000 nid=0x3b00 runnable [0x0000006c5c5fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084ac1ea0> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084ac1d60> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-request-timer[1]" #39 daemon prio=5 os_prio=0 cpu=15.63ms elapsed=544.33s tid=0x00000179bf68b800 nid=0x1550 waiting on condition [0x0000006c5c6ff000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep([email protected]/Native Method)
at io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"elastic-evictor-1" #50 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.15s tid=0x00000179bf692000 nid=0xcfc waiting on condition [0x0000006c5c7fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084af8098> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-6" #51 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.14s tid=0x00000179bf68c000 nid=0x34b4 runnable [0x0000006c5c8fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084acdde8> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084acdd60> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-7" #52 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=544.14s tid=0x00000179bf68d000 nid=0x2df0 runnable [0x0000006c5c9fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084ad1210> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084ad10d0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"elastic-2" #53 daemon prio=5 os_prio=0 cpu=31.25ms elapsed=544.11s tid=0x00000179c4e58000 nid=0x3670 waiting on condition [0x0000006c5cafe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084af16c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-8" #54 daemon prio=5 os_prio=0 cpu=31.25ms elapsed=543.78s tid=0x00000179c4e5c000 nid=0x900 runnable [0x0000006c5cbfe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084a32000> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084a31ed0> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-9" #55 daemon prio=5 os_prio=0 cpu=62.50ms elapsed=543.78s tid=0x00000179c4e59000 nid=0x2a7c waiting for monitor entry [0x0000006c5ccfc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.io.PrintStream.write([email protected]/PrintStream.java:558)
- waiting to lock <0x0000000081bee978> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes([email protected]/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implWrite([email protected]/StreamEncoder.java:303)
at sun.nio.cs.StreamEncoder.implWrite([email protected]/StreamEncoder.java:281)
at sun.nio.cs.StreamEncoder.write([email protected]/StreamEncoder.java:125)
- locked <0x0000000084aaef60> (a java.io.OutputStreamWriter)
at sun.nio.cs.StreamEncoder.write([email protected]/StreamEncoder.java:135)
at java.io.OutputStreamWriter.write([email protected]/OutputStreamWriter.java:229)
at java.io.Writer.write([email protected]/Writer.java:249)
at java.util.logging.StreamHandler.publish([email protected]/StreamHandler.java:212)
- locked <0x0000000084b3a860> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish([email protected]/ConsoleHandler.java:95)
at java.util.logging.Logger.log([email protected]/Logger.java:979)
at java.util.logging.Logger.doLog([email protected]/Logger.java:1006)
at java.util.logging.Logger.log([email protected]/Logger.java:1117)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._reportProblem(OptimizedBeanPropertyWriter.java:131)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._handleProblem(OptimizedBeanPropertyWriter.java:111)
- locked <0x0000000084aaf030> (a com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter)
at com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter.serializeAsField(ObjectFieldPropertyWriter.java:47)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1838)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:291)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:272)
at com.fasterxml.jackson.databind.SerializerProvider.defaultSerializeValue(SerializerProvider.java:1119)
at com.fasterxml.jackson.databind.node.POJONode.serialize(POJONode.java:115)
at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:39)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:20)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4409)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3663)
at com.azure.cosmos.CosmosException.toString(CosmosException.java:342)
at java.lang.Throwable.([email protected]/Throwable.java:316)
at java.lang.Exception.([email protected]/Exception.java:102)
at java.lang.RuntimeException.([email protected]/RuntimeException.java:96)
at java.util.concurrent.CompletionException.([email protected]/CompletionException.java:88)
at java.util.concurrent.CompletableFuture.encodeThrowable([email protected]/CompletableFuture.java:331)
at java.util.concurrent.CompletableFuture.completeThrowable([email protected]/CompletableFuture.java:346)
at java.util.concurrent.CompletableFuture.uniWhenComplete([email protected]/CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire([email protected]/CompletableFuture.java:837)
at java.util.concurrent.CompletableFuture.postComplete([email protected]/CompletableFuture.java:506)
at java.util.concurrent.CompletableFuture.completeExceptionally([email protected]/CompletableFuture.java:2088)
at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.messageReceived(RntbdRequestManager.java:856)
at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.channelRead(RntbdRequestManager.java:181)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-10" #56 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=543.74s tid=0x00000179c4e5d000 nid=0x3704 runnable [0x0000006c5cdfe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084add088> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084add000> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-11" #58 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=543.62s tid=0x00000179c4e5b800 nid=0x2e08 runnable [0x0000006c5cffe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084b70888> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084b70758> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-12" #59 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=543.56s tid=0x00000179c4e57800 nid=0x120 runnable [0x0000006c5d0fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084b68f28> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084b68df8> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-13" #60 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=543.52s tid=0x00000179c4e5e000 nid=0x2c90 runnable [0x0000006c5d1fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084b615c8> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084b61498> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"elastic-4" #62 daemon prio=5 os_prio=0 cpu=109.38ms elapsed=543.49s tid=0x00000179c4e5e800 nid=0x2124 waiting on condition [0x0000006c5d2fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000084aead90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-4" #64 daemon prio=5 os_prio=0 cpu=15.63ms elapsed=543.49s tid=0x00000179c4e59800 nid=0x3bac waiting on condition [0x0000006c5d4fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e1b38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-5" #65 daemon prio=5 os_prio=0 cpu=109.38ms elapsed=543.48s tid=0x00000179c4e66000 nid=0x3878 waiting on condition [0x0000006c5d5ff000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e1998> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-6" #66 daemon prio=5 os_prio=0 cpu=15.63ms elapsed=543.48s tid=0x00000179c4e62000 nid=0x3598 waiting on condition [0x0000006c5d6fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e17f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-7" #67 daemon prio=5 os_prio=0 cpu=62.50ms elapsed=543.39s tid=0x00000179c4e60800 nid=0x11b8 waiting on condition [0x0000006c5d7fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e1628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"parallel-8" #68 daemon prio=5 os_prio=0 cpu=78.13ms elapsed=543.38s tid=0x00000179c4e61000 nid=0x36b8 waiting on condition [0x0000006c5d8fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000000821e1488> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1170)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-14" #69 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=543.27s tid=0x00000179c4e65000 nid=0xa94 runnable [0x0000006c5d9fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084b59c68> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084b59b38> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"cosmos-rntbd-nio-2-15" #70 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=526.39s tid=0x00000179c4e62800 nid=0x3004 runnable [0x0000006c5affe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0([email protected]/Native Method)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll([email protected]/WindowsSelectorImpl.java:339)
at sun.nio.ch.WindowsSelectorImpl.doSelect([email protected]/WindowsSelectorImpl.java:167)
at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
- locked <0x0000000084b4b2c8> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x0000000084b4b198> (a sun.nio.ch.WindowsSelectorImpl)
at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:141)
at io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:810)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"ForkJoinPool.commonPool-worker-11" #71 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=186.94s tid=0x00000179c4e5a800 nid=0x39f4 waiting on condition [0x0000006c58aff000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000081c879c0> (a java.util.concurrent.ForkJoinPool)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at java.util.concurrent.ForkJoinPool.runWorker([email protected]/ForkJoinPool.java:1628)
at java.util.concurrent.ForkJoinWorkerThread.run([email protected]/ForkJoinWorkerThread.java:177)
Locked ownable synchronizers:
- None
"RMI TCP Accept-0" #72 daemon prio=5 os_prio=0 cpu=93.75ms elapsed=15.11s tid=0x00000179c4e63800 nid=0x3acc runnable [0x0000006c58cfe000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.accept0([email protected]/Native Method)
at java.net.PlainSocketImpl.socketAccept([email protected]/PlainSocketImpl.java:159)
at java.net.AbstractPlainSocketImpl.accept([email protected]/AbstractPlainSocketImpl.java:458)
at java.net.ServerSocket.implAccept([email protected]/ServerSocket.java:551)
at java.net.ServerSocket.accept([email protected]/ServerSocket.java:519)
at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept([email protected]/LocalRMIServerSocketFactory.java:52)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop([email protected]/TCPTransport.java:394)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run([email protected]/TCPTransport.java:366)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"RMI TCP Connection(1)-10.10.12.5" #73 daemon prio=5 os_prio=0 cpu=250.00ms elapsed=14.84s tid=0x00000179c4e64800 nid=0x37ec runnable [0x0000006c5a6fd000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0([email protected]/Native Method)
at java.net.SocketInputStream.socketRead([email protected]/SocketInputStream.java:115)
at java.net.SocketInputStream.read([email protected]/SocketInputStream.java:168)
at java.net.SocketInputStream.read([email protected]/SocketInputStream.java:140)
at java.io.BufferedInputStream.fill([email protected]/BufferedInputStream.java:252)
at java.io.BufferedInputStream.read([email protected]/BufferedInputStream.java:271)
- locked <0x0000000086ac9518> (a java.io.BufferedInputStream)
at java.io.FilterInputStream.read([email protected]/FilterInputStream.java:83)
at sun.rmi.transport.tcp.TCPTransport.handleMessages([email protected]/TCPTransport.java:544)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0([email protected]/TCPTransport.java:796)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0([email protected]/TCPTransport.java:677)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$881/0x000000010054d840.run([email protected]/Unknown Source)
at java.security.AccessController.doPrivileged([email protected]/Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run([email protected]/TCPTransport.java:676)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- <0x0000000086b8baf0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"RMI Scheduler(0)" #74 daemon prio=5 os_prio=0 cpu=0.00ms elapsed=14.80s tid=0x00000179c4ca2000 nid=0x3844 waiting on condition [0x0000006c5a9fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x0000000086ce1f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"JMX server connection timeout 75" #75 daemon prio=5 os_prio=0 cpu=31.25ms elapsed=14.78s tid=0x00000179c4ca0800 nid=0x1b2c in Object.wait() [0x0000006c5adff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <0x00000000869f94c0> (a [I)
at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run([email protected]/ServerCommunicatorAdmin.java:171)
- waiting to re-lock in wait() <0x00000000869f94c0> (a [I)
at java.lang.Thread.run([email protected]/Thread.java:834)
Locked ownable synchronizers:
- None
"VM Thread" os_prio=2 cpu=156.25ms elapsed=549.30s tid=0x00000179bce32800 nid=0x1260 runnable
"GC Thread#0" os_prio=2 cpu=125.00ms elapsed=549.32s tid=0x000001799c201000 nid=0x25dc runnable
"GC Thread#1" os_prio=2 cpu=125.00ms elapsed=548.53s tid=0x00000179be735800 nid=0x11ec runnable
"GC Thread#2" os_prio=2 cpu=156.25ms elapsed=548.53s tid=0x00000179be753000 nid=0x3188 runnable
"GC Thread#3" os_prio=2 cpu=93.75ms elapsed=545.66s tid=0x00000179c0442000 nid=0x3b2c runnable
"G1 Main Marker" os_prio=2 cpu=0.00ms elapsed=549.32s tid=0x000001799c232800 nid=0x1688 runnable
"G1 Conc#0" os_prio=2 cpu=15.63ms elapsed=549.32s tid=0x000001799c234000 nid=0x94 runnable
"G1 Conc#1" os_prio=2 cpu=15.63ms elapsed=547.33s tid=0x00000179c0457000 nid=0x3b8c runnable
"G1 Refine#0" os_prio=2 cpu=0.00ms elapsed=549.31s tid=0x00000179bcca2800 nid=0xa04 runnable
"G1 Young RemSet Sampling" os_prio=2 cpu=0.00ms elapsed=549.31s tid=0x000001799c2b8800 nid=0x33c4 runnable
"VM Periodic Task Thread" os_prio=2 cpu=93.75ms elapsed=548.67s tid=0x00000179be736000 nid=0x1dd4 waiting on condition
JNI global refs: 122, weak refs: 6455
Found one Java-level deadlock:
"main":
waiting to lock monitor 0x00000179bce60800 (object 0x0000000084b3a860, a java.util.logging.ConsoleHandler),
which is held by "cosmos-rntbd-nio-2-9"
"cosmos-rntbd-nio-2-9":
waiting to lock monitor 0x00000179c4cb5600 (object 0x0000000081bee978, a java.io.PrintStream),
which is held by "main"
Java stack information for the threads listed above:
"main":
at java.util.logging.StreamHandler.publish([email protected]/StreamHandler.java:194)
- waiting to lock <0x0000000084b3a860> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish([email protected]/ConsoleHandler.java:95)
at java.util.logging.Logger.log([email protected]/Logger.java:979)
at java.util.logging.Logger.doLog([email protected]/Logger.java:1006)
at java.util.logging.Logger.log([email protected]/Logger.java:1117)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._reportProblem(OptimizedBeanPropertyWriter.java:131)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._handleProblem(OptimizedBeanPropertyWriter.java:111)
- locked <0x0000000084b3aab8> (a com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter)
at com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter.serializeAsField(ObjectFieldPropertyWriter.java:47)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1838)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:291)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:272)
at com.fasterxml.jackson.databind.SerializerProvider.defaultSerializeValue(SerializerProvider.java:1119)
at com.fasterxml.jackson.databind.node.POJONode.serialize(POJONode.java:115)
at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:39)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:20)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4409)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3663)
at com.azure.cosmos.CosmosException.toString(CosmosException.java:342)
at java.lang.String.valueOf([email protected]/String.java:2951)
at java.io.PrintStream.println([email protected]/PrintStream.java:897)
at java.lang.Throwable$WrappedPrintStream.println([email protected]/Throwable.java:752)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:659)
- locked <0x0000000081bee978> (a java.io.PrintStream)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:648)
at java.lang.Throwable.printStackTrace([email protected]/Throwable.java:639)
at TestCosmos.main(TestCosmos.java:26)
"cosmos-rntbd-nio-2-9":
at java.io.PrintStream.write([email protected]/PrintStream.java:558)
- waiting to lock <0x0000000081bee978> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes([email protected]/StreamEncoder.java:233)
at sun.nio.cs.StreamEncoder.implWrite([email protected]/StreamEncoder.java:303)
at sun.nio.cs.StreamEncoder.implWrite([email protected]/StreamEncoder.java:281)
at sun.nio.cs.StreamEncoder.write([email protected]/StreamEncoder.java:125)
- locked <0x0000000084aaef60> (a java.io.OutputStreamWriter)
at sun.nio.cs.StreamEncoder.write([email protected]/StreamEncoder.java:135)
at java.io.OutputStreamWriter.write([email protected]/OutputStreamWriter.java:229)
at java.io.Writer.write([email protected]/Writer.java:249)
at java.util.logging.StreamHandler.publish([email protected]/StreamHandler.java:212)
- locked <0x0000000084b3a860> (a java.util.logging.ConsoleHandler)
at java.util.logging.ConsoleHandler.publish([email protected]/ConsoleHandler.java:95)
at java.util.logging.Logger.log([email protected]/Logger.java:979)
at java.util.logging.Logger.doLog([email protected]/Logger.java:1006)
at java.util.logging.Logger.log([email protected]/Logger.java:1117)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._reportProblem(OptimizedBeanPropertyWriter.java:131)
at com.fasterxml.jackson.module.afterburner.ser.OptimizedBeanPropertyWriter._handleProblem(OptimizedBeanPropertyWriter.java:111)
- locked <0x0000000084aaf030> (a com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter)
at com.fasterxml.jackson.module.afterburner.ser.ObjectFieldPropertyWriter.serializeAsField(ObjectFieldPropertyWriter.java:47)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1838)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:291)
at com.azure.cosmos.ClientSideRequestStatistics$ClientSideRequestStatisticsSerializer.serialize(ClientSideRequestStatistics.java:272)
at com.fasterxml.jackson.databind.SerializerProvider.defaultSerializeValue(SerializerProvider.java:1119)
at com.fasterxml.jackson.databind.node.POJONode.serialize(POJONode.java:115)
at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:39)
at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:20)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4409)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3663)
at com.azure.cosmos.CosmosException.toString(CosmosException.java:342)
at java.lang.Throwable.([email protected]/Throwable.java:316)
at java.lang.Exception.([email protected]/Exception.java:102)
at java.lang.RuntimeException.([email protected]/RuntimeException.java:96)
at java.util.concurrent.CompletionException.([email protected]/CompletionException.java:88)
at java.util.concurrent.CompletableFuture.encodeThrowable([email protected]/CompletableFuture.java:331)
at java.util.concurrent.CompletableFuture.completeThrowable([email protected]/CompletableFuture.java:346)
at java.util.concurrent.CompletableFuture.uniWhenComplete([email protected]/CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire([email protected]/CompletableFuture.java:837)
at java.util.concurrent.CompletableFuture.postComplete([email protected]/CompletableFuture.java:506)
at java.util.concurrent.CompletableFuture.completeExceptionally([email protected]/CompletableFuture.java:2088)
at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.messageReceived(RntbdRequestManager.java:856)
at com.azure.cosmos.implementation.directconnectivity.rntbd.RntbdRequestManager.channelRead(RntbdRequestManager.java:181)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1282)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1329)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run([email protected]/Thread.java:834)
Found 1 deadlock.
Code Snippet
CosmosAsyncContainer fs = ...;
EntryPOJO entry = new EntryPOJO();
entry.filename = path;
entry.data = value;
fs.createItem( entry ).block();
Setup (please complete the following information):
The text was updated successfully, but these errors were encountered: