Access NodePort at private IPv4 - equivalent of externalAccess.service.useHostIPs?

[brsolomon-deloitte]

In the Bitnami Kafka Helm chart there is a configuration parameter externalAccess.service.useHostIPs

Use service host IPs to configure Kafka external listener when service type is NodePort

What is the equivalent of this with Strimzi for kafka-external-bootstrap.

Currently:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: {{ .Values.kafka.version }}
    replicas: {{ .Values.kafka.replicas }}
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: nodeport
        tls: false
    config:
      auto.create.topics.enable: "true"
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      log.message.format.version: {{ printf "%s.%s" (split "." .Values.kafka.version)._0 (split "." .Values.kafka.version)._1 }}
      inter.broker.protocol.version: {{ printf "%s.%s" (split "." .Values.kafka.version)._0 (split "." .Values.kafka.version)._1 }}
    storage:
      type: ephemeral
  zookeeper:
    replicas: {{ .Values.zookeeper.replicas }}
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

This creates my-cluster-kafka-external-bootstrap on NodePort 32426.

The issue is that the connection seems to attempt to route over the public Internet. I do not want this; I want to access solely by private IPv4.

$ bin/kafka-topics.sh --create --if-not-exists --topic test99 --bootstrap-server 172.xx.xx.xx:32426
[2021-11-05 14:03:43,160] WARN [AdminClient clientId=adminclient-1] Connection to node 0 (ec2-18-xx-xx-xx.region.compute.amazonaws.com/18.xx.xx.xx:32509) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-05 14:03:43,202] WARN [AdminClient clientId=adminclient-1] Connection to node 1 (ec2-18-xx-xx-xx.region.compute.amazonaws.com/18.xx.xx.xx:30003) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-11-05 14:03:43,244] WARN [AdminClient clientId=adminclient-1] Connection to node 2 (ec2-18-xx-xx-xx.region.compute.amazonaws.com/18.xx.xx.xx:32612) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

It's clearly visible that in some way something is advertising its public IP/DNS and I want to avoid that. So what is the equivalent of externalAccess.service.useHostIPs for Strimzi's CRDs?

[brsolomon-deloitte]

Update: I see some mention of this in https://strimzi.io/blog/2019/04/23/accessing-kafka-part-2/#troubleshooting-node-ports

It seems to recommend the following:

listeners:
  # ...
  - name: external
    port: 9094
    type: nodeport
    tls: false
    configuration:
      brokers:
      - broker: 0
        advertisedHost: XXX.XXX.XXX.XXX
      - broker: 1
        advertisedHost: XXX.XXX.XXX.XXX
      - broker: 2
        advertisedHost: XXX.XXX.XXX.XXX

Is there no cleaner way to do this? Needing to explicitly specify IP addresses seems like a huge anti-pattern.

The article mentions

To get the node address, the init container has to talk with the Kubernetes API and get the node resource. In the status of the node resource, the address is normally listed as one of the following:

• External DNS
• External IP
• Internal DNS
• Internal IP
• Hostname

Is there a way to 'cross off' the first 3 and tell the init container to use the internal ip?

If the answer is no, this would basically be a nonstarter for using strimzi unfortunately.

[brsolomon-deloitte]

Answering my own question:

use spec.kafka.listeners[x].configuration. preferredNodePortAddressType = InternalIP

spec:
  kafka:
    version: {{ .Values.kafka.version }}
    replicas: {{ .Values.kafka.replicas }}
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: nodeport
        tls: false
        configuration:
          bootstrap:
            nodePort: 31094
          preferredNodePortAddressType: InternalIP

Description:

   preferredNodePortAddressType	<string>
     Defines which address type should be used as the node address. Available
     types are: `ExternalDNS`, `ExternalIP`, `InternalDNS`, `InternalIP` and
     `Hostname`. By default, the addresses will be used in the following order
     (the first one found will be used):

     * `ExternalDNS`
     * `ExternalIP`
     * `InternalDNS`
     * `InternalIP`
     * `Hostname`

     This field is used to select the preferred address type, which is checked
     first. If no address is found for this address type, the other types are
     checked in the default order. This field can only be used with `nodeport`
     type listener.

[scholzj]

Yeah, that is the right way to do it 👍