Strimzi
Replies: 1 comment 3 replies
-
|
The resources in your MM2 are IMHO misconfigured. It will not work with the resource request and our JVM settings do not match it ether. You should give it proper resources sufficient for it to run. The timeout error suggests some general connection issue or the Kafka cluster being non-repsonsive rather than some permission issues. But you can doublecheck that in the broker logs which would clearly list any permission issues including the exact details. So I think that is where you should look. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the reply Jakub :) You are right this is most likely a resource issue. Admin operations on topics are infamously slow. I tried setting these configs on MM2, but to no use unfortunately... - alias: "cluster-target"
bootstrapServers: kafka-mirror-kafka-bootstrap.default.svc:9092
authentication:
type: scram-sha-512
username: kafka-mm2-target-user
passwordSecret:
password: password
secretName: kafka-mm2-target-user
config:
request.timeout.ms: 180000
metadata.max.age.ms: 180000
default.api.timeout.ms: 180000 |
Beta Was this translation helpful? Give feedback.
-
|
As I said, I'm not sure the issue is caused by the resources. The timeout could be also caused by some other issue. The configuration obviously does not really say how many resources it gets. Just that it will be between 10m CPU / 124Mi RAM and 750m CPU / 512Mi RAM. How much it actually is will depend on your cluster. I would not expect it to work with 10m CPU / 124Mi RAM. But it would probably work with 750m CPU / 512Mi RAM. So it depends. I'm afraid I do not know how to configure these timeouts -> and with 10mCPU / 124Mi RAM it will likely not help anyway. |
Beta Was this translation helpful? Give feedback.
-
|
Got you. Will try to see if I can find a way to override the timeouts, otherwise I'll just throw more hardware at it. Thank you for your time Jakub :) |
Beta Was this translation helpful? Give feedback.
-
Hello :)
I am trying to ensure I can utilize MM2 in a production environment for an important client, unfortunately I am falling short somewhere.
Context
I am deploying both my source and target clusters on GKE 1.27.8-gke.1067004
I am using version 0.40 of the Strimzi operator and version 3.7.0 for both Kafka clusters (of which I won't include the configurations to avoid making the message too long. If needed, I will of course provide them)
Mirror Maker 2 configuration
Note: I am using a global private load balancer to access the source cluster from a different GCP region
One thing to note, is that for both clusters I am basically granting superuser privileges. Which should therefore grant permissions for all operations.
However something is not working, as I am getting this error
I also attached a zip compressed complete extract of the pod logs mm2.zip
For the life of me I truly cannot understand what is going on here... How is the operation failing, when the kafkauser has all the required permissions and the cluster is already accessible?
On the brokers' side, I obviously checked the logs, and there is nothing that hints at any possible errors/misconfigurations. Consider that I am running the logging in TRACE mode and the only information I am getting is about rebalancing and assignments, nothing special.
Is there any blatant misconfiguration on my part? What is wrong here?
Thank you in advance :)
Beta Was this translation helpful? Give feedback.
All reactions