Kubernetes Ingress Controller Fake Certificate

[adityakrk]

Hi we have a two types of Nginx Controller
1)nginxexternal -> we can access outside the cluster .
2)nginxinternal -> we can access inside the cluster and with in the company network

This is the kafka.yaml file which we are using

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  labels:
    {{- include "ccp-infra-kafka.labels" . | nindent 4 }}
spec:
  kafka:
    version: 3.6.0
    replicas: {{int .Values.kafka.replicas }}
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: ingress
        tls: true
        configuration:
           brokerCertChainAndKey:
             certificate: tls.crt
             key: tls.key
             secretName: kafka-cluster-lets-encrypt
          bootstrap:
            annotations:
              kubernetes.io/ingress.class: nginxinternal
            host:  kafka-cluster-bootstrap.{{ .Release.Namespace }}.{{ .Values.global.clusterName }}.{{ .Values.global.clusterDomain }}
          brokers:
            {{- $namespace := .Release.Namespace }}
            {{- $clusterName := .Values.global.clusterName }}
            {{- $clusterDomain := .Values.global.clusterDomain }}
            {{- range $index, $element := until (int .Values.kafka.replicas) }}
            - broker: {{ $index }}
              host: kafka-cluster-kafka-{{ $index }}.{{ $namespace }}.{{ $clusterName }}.{{ $clusterDomain }}
              annotations:
                kubernetes.io/ingress.class: nginxinternal
            {{- end }}
        class: nginxinternal
    config:
      default.replication.factor: 3
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.6"
      auto.create.topics.enable: true
    storage:
      type: jbod
      volumes:
        - id: 0
          type: persistent-claim
          size: 10Gi
          deleteClaim: false
    {{- if .Values.monitoring.enabled }}
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
    {{- end }}
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 10Gi
      deleteClaim: false
    {{- if .Values.monitoring.enabled }}
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: zookeeper-metrics-config.yml
    {{- end }}
  entityOperator:
    topicOperator: {}
    userOperator: {}
  {{- if .Values.monitoring.kafkaExporter.enabled }}
  kafkaExporter: {}
  {{- end }}

i am using nginxinternal when i try to access bootstrap url it is showing Fake certificate so couldn't able to connect and the same above kafka configuration will work for nginxexternal .
1)nginxexternalcontroller -> has a --enable-ssl-passthrough args.
2)nginxinternalcontroller ->doesn't have --enable-ssl-passthrough args.

And this is the certificate code

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kafka-cluster-lets-encrypt
spec:
  secretName: kafka-cluster-lets-encrypt
  issuerRef:
    name: letsencrypt-dns-staging
    kind: ClusterIssuer
    group: cert-manager.io
  subject:
    organizations:
      - my-org
  dnsNames:
    - kafka-cluster-bootstrap.xxxxxxxxx.hcvpc.io
    - kafka-cluster-kafka-0.xxxxxxxxx.hcvpc.io
    - kafka-cluster-kafka-1.xxxxxxxxx.hcvpc.io
    - kafka-cluster-kafka-2.xxxxxxxxx.hcvpc.io

[scholzj]

I guess you answered it yourself:

nginxinternalcontroller ->doesn't have --enable-ssl-passthrough args.

You need to enable the TLS passthrough for this to work. This is required for use with Kafka because Kafka does not use HTTP protocol. And without it being enabled, you get the Fake certificate because the connection terminates in the Ingress instead of being passed through to the Kafka broker(s).

[adityakrk]

Is there any option where we can generate certificate using cert manager and terminate at the ingress level ?

[scholzj]

No, there is no such option. As I said, the Kafka protocol is not HTTP based and would not pass through ingress if it wasn't for the TLS passthrough

[adityakrk]

Ok got it thanks for quick reply .

[adityakrk]

Hi
any idea how can we have tls secret ? For now it is blank

[scholzj]

You cannot - as I said, you have to use TLS passthrough for Kafka. So any secrets at the Ingress level make no sense.