Replies: 2 comments
-
Maybe you can format the log properly to make it readable? You should probably also share the full log and not just single exception. Maybe you should also explain how your API server works. Why does it close the connection exactly in this place and not in the many steps before? Also, keep in mind that we do not support Kubernetes 1.22 anymore. |
Beta Was this translation helpful? Give feedback.
-
Triaged on the community call on 2.5.2024: There does not seem to be a log describing the issue and itis not clear if it affects supported environments. This should be converted to discussion and we can continue there. |
Beta Was this translation helpful? Give feedback.
-
Bug Description
when the k8s apiserver use--goaway-chance flag, apiserver will randomly close the client http2 connection(GOAWAY),but fabric8 client do not retry. this make kafka reconciliation failed。 the failed log is:
[2024-04-22 17:15:14]
content: 2024-04-22 17:15:14 WARN VertxUtil:103 - Reconciliation #2947(timer) Kafka(strimzi-kafka/kafka35t3p0m8406): Caught exception while waiting for Pods resource kafka35t3p0m8406-1077-3000-3000 in namespace strimzi-kafka to get ready
-23
[2024-04-22 17:15:14]
content: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Pod] with name: [kafka35t3p0m8406-1077-3000-3000] in namespace: [strimzi-kafka] failed.
-22
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:159) ~[io.fabric8.kubernetes-client-api-6.9.2.jar:?]
-21
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.requireFromServer(BaseOperation.java:194) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-20
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.get(BaseOperation.java:148) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-19
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.isReady(BaseOperation.java:912) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-18
[2024-04-22 17:15:14]
content: at io.strimzi.operator.common.operator.resource.AbstractReadyNamespacedResourceOperator.isReady(AbstractReadyNamespacedResourceOperator.java:67) ~[io.strimzi.operator-common-0.39.0-0.0.2.jar:0.39.0-0.0.2]
-17
[2024-04-22 17:15:14]
content: at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.lambda$waitFor$7(AbstractNamespacedResourceOperator.java:393) ~[io.strimzi.operator-common-0.39.0-0.0.2.jar:0.39.0-0.0.2]
-16
[2024-04-22 17:15:14]
content: at io.strimzi.operator.common.VertxUtil$1.lambda$handle$0(VertxUtil.java:101) ~[io.strimzi.operator-common-0.39.0-0.0.2.jar:0.39.0-0.0.2]
-15
[2024-04-22 17:15:14]
content: at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$0(ContextImpl.java:177) ~[io.vertx.vertx-core-4.5.0.jar:4.5.0]
-14
[2024-04-22 17:15:14]
content: at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:276) ~[io.vertx.vertx-core-4.5.0.jar:4.5.0]
-13
[2024-04-22 17:15:14]
content: at io.vertx.core.impl.ContextImpl.lambda$internalExecuteBlocking$2(ContextImpl.java:209) ~[io.vertx.vertx-core-4.5.0.jar:4.5.0]
-12
[2024-04-22 17:15:14]
content: at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[io.vertx.vertx-core-4.5.0.jar:4.5.0]
-11
[2024-04-22 17:15:14]
content: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
-10
[2024-04-22 17:15:14]
content: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
-9
[2024-04-22 17:15:14]
content: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty.netty-common-4.1.100.Final.jar:4.1.100.Final]
-8
[2024-04-22 17:15:14]
content: at java.lang.Thread.run(Thread.java:840) ~[?:?]
-7
[2024-04-22 17:15:14]
content: Caused by: java.io.IOException: /192.168.176.105:55338: GOAWAY received
-6
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:504) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-5
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:524) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-4
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleGet(OperationSupport.java:467) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-3
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleGet(BaseOperation.java:791) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-2
[2024-04-22 17:15:14]
content: at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.requireFromServer(BaseOperation.java:192) ~[io.fabric8.kubernetes-client-6.9.2.jar:?]
-1
[2024-04-22 17:15:14]
content: ... 13 more
0
[2024-04-22 17:15:14]
content: Caused by: java.io.IOException: /192.168.176.105:55338: GOAWAY received
+1
Steps to reproduce
No response
Expected behavior
No response
Strimzi version
0.39.0
Kubernetes version
Kubernetes 1.22
Installation method
No response
Infrastructure
No response
Configuration files and logs
No response
Additional context
No response
Beta Was this translation helpful? Give feedback.
All reactions