Caught exception while creating Deployment entity-operator #8154

marcoadasilvaa · 2023-02-24T13:53:55Z

marcoadasilvaa
Feb 24, 2023

Describe the bug
Clear installation of the strimzi operator in a new cluster fails because can't create or update operator entity deployment. Maybe is related to a no connection of the API server to create this resource but strimzi can create another resource, not this.
Operator and Cluster stay in the same namespace.

To Reproduce
Steps to reproduce the behavior:

Deploy strimzi/strimzi-kafka-operator 0.33.2
Deploy Custom Resource Kafka Cluster
Waiting for the ready state of Zookeper and Broker
Check state of kafka cluster on CRD kafka.io.strimzi.io -> kafka

See the error on the operator pod related to entity deployment:

2023-02-24T12:53:04.824016725Z 2023-02-24 12:53:04 DEBUG AbstractNamespacedResourceOperator:106 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Deployment kafka-labs/labs-cluster-entity-operator does not exist, creating it
2023-02-24T12:53:14.826847204Z 2023-02-24 12:53:14 DEBUG AbstractNamespacedResourceOperator:262 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Caught exception while creating Deployment labs-cluster-entity-operator in namespace kafka-labs
2023-02-24T12:53:14.826847204Z io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:129) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:122) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:44) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:1107) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.internalCreate(AbstractNamespacedResourceOperator.java:258) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.826847204Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.lambda$reconcile$0(AbstractNamespacedResourceOperator.java:107) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.826847204Z 	at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.826847204Z 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.826847204Z 	at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.826847204Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.826847204Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.826847204Z 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty.netty-common-4.1.86.Final.jar:4.1.86.Final]
2023-02-24T12:53:14.826847204Z 	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2023-02-24T12:53:14.826847204Z Caused by: java.io.IOException: timeout
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:533) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:570) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:554) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleCreate(OperationSupport.java:347) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:704) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:42) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.826847204Z 	... 11 more
2023-02-24T12:53:14.826847204Z Caused by: java.net.SocketTimeoutException: timeout
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:672) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:680) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:153) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:131) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:201) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.826847204Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.826847204Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.826847204Z 	... 1 more
2023-02-24T12:53:14.827694203Z 2023-02-24 12:53:14 ERROR AbstractOperator:260 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): createOrUpdate failed
2023-02-24T12:53:14.827694203Z io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:129) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:122) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:44) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:1107) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.internalCreate(AbstractNamespacedResourceOperator.java:258) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.827694203Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.lambda$reconcile$0(AbstractNamespacedResourceOperator.java:107) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.827694203Z 	at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.827694203Z 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.827694203Z 	at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.827694203Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.827694203Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.827694203Z 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty.netty-common-4.1.86.Final.jar:4.1.86.Final]
2023-02-24T12:53:14.827694203Z 	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2023-02-24T12:53:14.827694203Z Caused by: java.io.IOException: timeout
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:533) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:570) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:554) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleCreate(OperationSupport.java:347) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:704) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:42) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.827694203Z 	... 11 more
2023-02-24T12:53:14.827694203Z Caused by: java.net.SocketTimeoutException: timeout
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:672) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:680) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:153) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:131) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:201) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.827694203Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.827694203Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.827694203Z 	... 1 more
2023-02-24T12:53:14.874211779Z 2023-02-24 12:53:14 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:14.874274328Z 2023-02-24 12:53:14 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:53:14.875647138Z 2023-02-24 12:53:14 DEBUG StatusDiff:42 - Ignoring Status diff {"op":"replace","path":"/conditions/0/lastTransitionTime","value":"2023-02-24T12:53:14.826948747Z"}
2023-02-24T12:53:14.875831568Z 2023-02-24 12:53:14 DEBUG AbstractOperator:337 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Status did not change
2023-02-24T12:53:14.882532867Z 2023-02-24 12:53:14 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:14.882561662Z 2023-02-24 12:53:14 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:53:14.883620725Z 2023-02-24 12:53:14 DEBUG AbstractOperator:560 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Removed metric strimzi.resource.statekafka-labs:Kafka/labs-cluster
2023-02-24T12:53:14.883864699Z 2023-02-24 12:53:14 DEBUG AbstractOperator:568 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Updated metric strimzi.resource.state[tag(kind=Kafka),tag(name=labs-cluster),tag(reason=An error has occurred.),tag(resource-namespace=kafka-labs)] = 0
2023-02-24T12:53:14.884297010Z 2023-02-24 12:53:14 WARN  AbstractOperator:525 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Failed to reconcile
2023-02-24T12:53:14.884297010Z io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:129) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:122) ~[io.fabric8.kubernetes-client-api-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:44) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:1107) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.internalCreate(AbstractNamespacedResourceOperator.java:258) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.884297010Z 	at io.strimzi.operator.common.operator.resource.AbstractNamespacedResourceOperator.lambda$reconcile$0(AbstractNamespacedResourceOperator.java:107) ~[io.strimzi.operator-common-0.33.2.jar:0.33.2]
2023-02-24T12:53:14.884297010Z 	at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.884297010Z 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.884297010Z 	at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135) ~[io.vertx.vertx-core-4.3.7.jar:4.3.7]
2023-02-24T12:53:14.884297010Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.884297010Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.884297010Z 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty.netty-common-4.1.86.Final.jar:4.1.86.Final]
2023-02-24T12:53:14.884297010Z 	at java.lang.Thread.run(Thread.java:833) ~[?:?]
2023-02-24T12:53:14.884297010Z Caused by: java.io.IOException: timeout
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:533) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:570) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:554) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleCreate(OperationSupport.java:347) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:704) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleCreate(BaseOperation.java:93) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	at io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:42) ~[io.fabric8.kubernetes-client-6.3.1.jar:?]
2023-02-24T12:53:14.884297010Z 	... 11 more
2023-02-24T12:53:14.884297010Z Caused by: java.net.SocketTimeoutException: timeout
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:672) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:680) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:153) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:131) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:88) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:201) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[com.squareup.okhttp3.okhttp-3.12.12.jar:?]
2023-02-24T12:53:14.884297010Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
2023-02-24T12:53:14.884297010Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
2023-02-24T12:53:14.884297010Z 	... 1 more
2023-02-24T12:53:14.884433320Z 2023-02-24 12:53:14 DEBUG AbstractOperator:447 - Reconciliation #78(watch) Kafka(kafka-labs/labs-cluster): Lock lock::kafka-labs::Kafka::labs-cluster released
2023-02-24T12:53:34.185237904Z 2023-02-24 12:53:34 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:34.185309179Z 2023-02-24 12:53:34 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:53:35.112120338Z 2023-02-24 12:53:35 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:35.112187817Z 2023-02-24 12:53:35 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:53:36.096996103Z 2023-02-24 12:53:36 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:36.097064874Z 2023-02-24 12:53:36 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:53:36.763102535Z 2023-02-24 12:53:36 DEBUG CustomResource:165 - Calling CustomResource#setApiVersion doesn't do anything because the API version is computed and shouldn't be changed
2023-02-24T12:53:36.763184811Z 2023-02-24 12:53:36 DEBUG CustomResource:176 - Calling CustomResource#setKind doesn't do anything because the Kind is computed and shouldn't be changed
2023-02-24T12:54:04.666769058Z 2023-02-24 12:54:04 DEBUG Config:630 - Trying to configure client from Kubernetes config...

Expected behavior
Cluster state ready with zookeeper, broker and operator entity

Environment (please complete the following information):

Strimzi version: 0.33.2
Installation method: helm install strimzi/strimzi-kafka-operator
Kubernetes cluster: 1.24.1
Infrastructure: OCI

YAML files and logs

Attach or copy paste the custom resources you used to deploy the Kafka cluster and the relevant YAMLs created by the Cluster Operator.

# Source: strimzi/templates/cluster.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
 name: labs-cluster
spec:
 kafka:
   replicas: 3
   listeners:
     - name: plain
       port: 9092
       type: internal
       tls: false
     - name: tls
       port: 9093
       type: internal
       tls: true
       authentication:
         type: tls
     - name: external
       port: 9094
       type: nodeport
       tls: false
   storage:
     type: jbod
     volumes:
     - id: 0
       type: persistent-claim
       size: 200Gi
       deleteClaim: false
   config:
     offsets.topic.replication.factor: 3
     transaction.state.log.replication.factor: 3
     transaction.state.log.min.isr: 2
     default.replication.factor: 3
     min.insync.replicas: 2
 zookeeper:
   replicas: 3
   storage:
     type: persistent-claim
     size: 50Gi
     deleteClaim: false
 entityOperator:
   topicOperator: {}
   userOperator: {}

Attach or copy and paste also the relevant logs.

Full log of operator strimzi

strimzi-cluster-operator-698b4dd66d-kf9pq.log

scholzj · 2023-02-24T15:13:34Z

scholzj
Feb 24, 2023
Maintainer

That suggests some issues with your Kubernetes API server where the request times out. This could be in the API server itself, but since other operations are passing, it could be also caused by some custom tooling you use. E.g. some webhooks etc. The best start might be to look into your Kubernetes logs if they contain some related information.

0 replies

marcoadasilvaa · 2023-02-24T18:58:55Z

marcoadasilvaa
Feb 24, 2023
Author

Thank you very much for your response, verifying our API server provider returns the HTTP 504 answer after 10 seconds. It is so strange because the other resources created or patches without problems, only fails with operator-entity deployment

1 reply

scholzj Feb 24, 2023
Maintainer

I would check if you have some Admission Webhooks watching for Deployments. They are often causing things like this.

romikps · 2024-01-23T10:35:34Z

romikps
Jan 23, 2024

We ran into the same issues and noticed that it didn't happen when we deployed our Kafka cluster in another k8s cluster. The problem was that our original k8s cluster was a bit slow, so the Kubernetes client didn't get a reply within the default timeout of 2000ms. Raising the timeout solved the problem. We had to set the following env variables when deploying the cluster operator with the Helm chart:

extraEnvs:
  - name: KUBERNETES_REQUEST_TIMEOUT
    value: "20000"
  - name: KUBERNETES_REQUEST_RETRIES
    value: "2"

Hope this helps in case it's still relevant for you.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strimzi

Caught exception while creating Deployment entity-operator #8154

{{title}}

Replies: 3 comments 1 reply

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Strimzi

Caught exception while creating Deployment entity-operator #8154

marcoadasilvaa Feb 24, 2023

Replies: 3 comments · 1 reply

scholzj Feb 24, 2023 Maintainer

marcoadasilvaa Feb 24, 2023 Author

scholzj Feb 24, 2023 Maintainer

romikps Jan 23, 2024

marcoadasilvaa
Feb 24, 2023

Replies: 3 comments 1 reply

scholzj
Feb 24, 2023
Maintainer

marcoadasilvaa
Feb 24, 2023
Author

scholzj Feb 24, 2023
Maintainer

romikps
Jan 23, 2024