Kafka pods not running in different nodes with pod-antiaffinity specified

[shreyasarani23]

Below is my kafka configuration I have used pod antiaffinity to schedule kafka pods on different nodes
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: my-kafka-cluster
labels:
app: kafkaPod
spec:
kafka:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- kafkPod
topologyKey: "kubernetes.io/hostname"
version: 2.7.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
log.message.format.version: "2.7"
auto.create.topics.enable: true
auto.leader.rebalance.enable: true
delete.topic.enable: true
storage:
type: persistent-claim
size: 1Gi
class: kafka-storage-class
deleteClaim: true
zookeeper:
replicas: 3
storage:
type: persistent-claim
size: 1Gi
class: kafka-storage-class
deleteClaim: true
entityOperator:
topicOperator: {}

But the problem is that only one kafka pod is getting deployed and the other two are in pending state.
When I describe the pod to check why it is in pending state it says the following
0/3 nodes are available: didn't match pod affinity/anti-affinity.

Is there anything wrong in my deployment file?
Please correct me If I am wrong

[scholzj]

Please check the documentation for where the affinity should be configured. I don't think you have it in the right place.

[scholzj]

Well, the error means what it means ... you said that the Kafka pods should not run on the same node as other pods with the label app: kafkaPod. But you do not have enough nodes to to achieve this. So you either need to change the requiredDuringScheduling rule to preferredDuringScheduling or us a different label. Keep in mind, that they way you configured it ZooKeeper will have this label as well.

[shreyasarani23]

Ohk I got it So what should I do now If want to run kafka pods on different nodes?
How can I give different labels so that kafka pods don't run on same nodes?

[scholzj]

You need to use different label just for the Kafka brokers. You can use the .spec.kafka.template... section in the Kafka CR for it. Or you can use one of the built-in Strimzi labels (do not know the name out of my head - you would need to check on the pods).

[shreyasarani23]

Can U share me the link where they have used different label for kafka brokers under .spec.kafka.template....
And how do u get the information about the in-built strimzi labels??

[scholzj]

I do not have the link at hand. But in the Using guide, search for Customizing Kubernetes Resources. The built-in pod labels are probably easiest to find by deploying the KAfka cluster without the affinity first.