how Kafka broker can be used without exposing the endpoints/port for each

[Subodh-create]

I have deployed cluster using this configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  namespace: kafka
spec:
  kafka:
    version: 3.5.1
    replicas: 3
    authorization:
     type: simple
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        authentication:
          type: scram-sha-512
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        tls: false
        type: cluster-ip
        configuration:
          bootstrap:
            alternativeNames:
               - kafka-bootstrap.example.com
          brokers:
          - broker: 0
            advertisedHost: kafka-broker-01.example.com
            advertisedPort: 9095
          - broker: 1
            advertisedHost: kafka-broker-02.example.com
            advertisedPort: 9096
          - broker: 2
            advertisedHost: kafka-broker-03.example.com
            advertisedPort: 9097
        authentication:
          type: scram-sha-512

    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: true
    config:
      sasl.enabled.mechanisms: scram-sha-512
      sasl.mechanism.inter.broker.protocol: scram-sha-512
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 2
      transaction.state.log.min.isr: 3
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.5.1"
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 20Gi
      deleteClaim: true
  entityOperator:
    topicOperator: {}
    userOperator: {}

After this all three broker service expose in 9094
For connecting client through ingress endpoint and i have open tcp in ingress

tcp:
  9094: kafka/kafka-cluster-kafka-external-bootstrap:9094
  9095: kafka/kafka-cluster-kafka-0:9094
  9096: kafka/kafka-cluster-kafka-1:9094
  9097: kafka/kafka-cluster-kafka-2:9094

i have put host in etc/hosts

172.19.254.201  kafka-bootstrap.example.com
172.19.254.201  kafka-broker-01.example.com
172.19.254.201  kafka-broker-02.example.com
172.19.254.201  kafka-broker-03.example.com

This work fine i am able to consume and produce message from this command

./kafka_2.13-3.7.0/bin/kafka-console-producer.sh --topic test-topic  --bootstrap-server kafka-bootstrap.example.com:9094  --producer.config client.properties 

it is working fine fine

That's all for the local setup, and I have also set up the same in an AKS cluster. However, we need to open tcp ports for all brokers to produce and consume messages like 9095 9096 and 9097 too. For production we need hundred of brokers how can i setup so that no need to open tcp port.
my question is how can we communication without opening all tcp ports

[scholzj]

Well, this is purely your decision to do it as you described. So if you don't want to do it that way, you can use one of the other ways supported by Strimzi. For example:

• Use a load balancers
• Use Ingress with TLS passhtorugh on a single port for all nodes and TLS-SNI
• Use Node Ports

[Subodh-create]

I am using a load balancer with the 'advertisedhost' to connect Kafka with the Ingress. Can I use the 'advertisedhost' only for the bootstrap, not for all brokers? Even if I use the 'advertisedhost' for all brokers, it still works fine but need to add all the brokers host in /etc/hosts. . I need the host only for the bootstrap, and that bootstrap should connect internally with brokers so that no need to specify all the brokers in hosts file.
@scholzj

[scholzj]

I am using a load balancer with the 'advertisedhost' to connect Kafka with the Ingress.

This does not really make any sense. You either use Ingress or Load Balancers.

Can I use the 'advertisedhost' only for the bootstrap, not for all brokers? Even if I use the 'advertisedhost' for all brokers, it still works fine but need to add all the brokers host in /etc/hosts. . I need the host only for the bootstrap, and that bootstrap should connect internally with brokers so that no need to specify all the brokers in hosts file

No, that is not how Kafka works. You need access to each broker directly.

[Subodh-create]

Thankyou @scholzj i got the answer