[Kafka Connect] Behaviour on limited resource in k8s

[vladislav-bakanov]

Hi
I have Strimzi Kafka Connect deployed and about 20 connectors. I specified only requested resources to 8Gb RAM per each replica, but I see that deployment started to consume more resources after some time. For example it grown from 2Gb to 20Gb RAM in one day.

Questions:

1. Should I specify limits for the deployment?
2. Why it grows infinitively? Does Kafka Connect caches something in memory?
3. What will happen if I specify limits and it will reach out the limit? PODs will be restarted, right?

[scholzj]

If you specified only the memory request, it means that in general the limit is unlimited. So it can grow as long as there is free memory and Java in general will tend to use it. Ideally, you should set both the limit and request to the same value (or if you set only the limit, it will also be used as a request).

What will happen if I specify limits and it will reach out the limit? PODs will be restarted, right?

The Pod will be killed in such a case. But keep in mind that Java has awareness of the available memory and just because now you see it consuming 20Gi does not mean it would run out of memory if you set the limit to 8Gi.

[vladislav-bakanov]

Thank you so much 🫶