You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
16:52:09 - The ZK session has expired CuratorEventImpl
16:52:09 - Curator connection state changed to LOST and Runtime halter is called probably on an unrecoverable error. Stopping the VM.
and the bug is - when ZK is already in a bad state the Astra close protocol should not try to make a ZK fetch like it does today. The ZK operations fails and and the exception ends up calling the Runtime falter again
Another side note, when this happens then in the next 20 seconds we see 450+ log messages with CuratorCache error like this
java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:821)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkState(CuratorFrameworkImpl.java:457)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:491)
at org.apache.curator.framework.recipes.cache.CuratorCacheImpl.nodeChanged(CuratorCacheImpl.java:266)
The text was updated successfully, but these errors were encountered:
bryanlb
changed the title
[BUG] ZK session expiry shuts down kaldb. Don't make ZK updates in the shutdown path
ZK session expiry shuts down kaldb. Don't make ZK updates in the shutdown path
Oct 30, 2023
bryanlb
changed the title
ZK session expiry shuts down kaldb. Don't make ZK updates in the shutdown path
ZK session expiry shuts down Astra. Don't make ZK updates in the shutdown path
Mar 11, 2024
To Reproduce
I noticed this in our production cluster
and the bug is - when ZK is already in a bad state the Astra close protocol should not try to make a ZK fetch like it does today. The ZK operations fails and and the exception ends up calling the Runtime falter again
Another side note, when this happens then in the next 20 seconds we see 450+ log messages with CuratorCache error like this
The text was updated successfully, but these errors were encountered: