You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello,
We are running into zookeeper issues while upgrading kafka using the operator from 2.8.0 -> 3.0.0
Our setup includes 3 zookeepers and 3 kafka brokers with below config (trimmed)
While upgrading the kafka version to 3.0.0:
stimzi operator detects zookeper vesion upgrade is required and starts rolling restarts,
1 zookeeper instance was upgraded fine,
2 zookeeper instance starts and eventually throws the below log and keeps restarting:
2024-01-07 07:48:40,934 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
2024-01-07 07:48:40,934 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
2024-01-07 07:48:40,936 INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [main]
2024-01-07 07:48:40,936 INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase) [main]
2024-01-07 07:48:41,233 INFO Using TLS encrypted quorum communication (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,233 INFO Port unification disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,233 INFO multiAddress.enabled set to false (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,234 INFO multiAddress.reachabilityCheckEnabled set to true (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,234 INFO multiAddress.reachabilityCheckTimeoutMs set to 1000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,234 INFO QuorumPeer communication is not secured! (SASL auth disabled) (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,234 INFO quorum.cnxn.threads.size set to 20 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2024-01-07 07:48:41,235 INFO Reading snapshot /var/lib/zookeeper/data/version-2/snapshot.600026f9b (org.apache.zookeeper.server.persistence.FileSnap) [main]
2024-01-07 07:48:42,535 WARN Got EOF exception while reading the digest, likely due to the reading an older snapshot. (org.apache.zookeeper.server.DataTree) [main]
Eventually after some restarts all 3 zookeeper instances enter unhealthy state and keep restarting
Can you please advise how to proceed from here?
Zookeeper data directory1:
2#
3#
Strimzi operator version: 0.27.1
K8s EKS 1.24
Steps to reproduce
Install 2.8.0 kafka using crd
Upgrade crd to kafka 3.0.0
Bug Description
Hello,
We are running into zookeeper issues while upgrading kafka using the operator from 2.8.0 -> 3.0.0
Our setup includes 3 zookeepers and 3 kafka brokers with below config (trimmed)
While upgrading the kafka version to 3.0.0:
1 zookeeper instance was upgraded fine,
2 zookeeper instance starts and eventually throws the below log and keeps restarting:
Eventually after some restarts all 3 zookeeper instances enter unhealthy state and keep restarting
Can you please advise how to proceed from here?
Zookeeper data directory1:
2#
3#
Strimzi operator version: 0.27.1
K8s EKS 1.24
Steps to reproduce
Install 2.8.0 kafka using crd
Upgrade crd to kafka 3.0.0
Expected behavior
No response
Strimzi version
0.27.1
Kubernetes version
EKS 1.24
Installation method
Helm
Infrastructure
Amazon EKS
Configuration files and logs
Additional context
No response
The text was updated successfully, but these errors were encountered: