Kafka SSL Connection Failure Over IPv6

[wi7sonjoseph]

Discussed in #1290

^{Originally posted by wi7sonjoseph September 10, 2024}
We have our Kafka cluster configured in SSL mode. Below are few of the details:

From Kafka config (server):

advertised.listeners=SSL://[2001:db8:2:2000:169::18]:9092,EXTERNAL_SSL://192.168.169.18:29092

From Certificate:

CN=2001:db8:2:2000:169::18

SubjectAlternativeName [
  DNSName: kafka01-server-1
  IPAddress: 192.168.169.18
  IPAddress: 2001:db8:2:2000:169:0:0:18
]

Issue:

When using IPv6, I am not able to write to Kafka (see below producer.properties).

Producer Properties:

bootstrap.servers=[2001:db8:2:2000:169::18]:9092

security.protocol=SSL
ssl.key.location=/opt/miq/cvs/ssl/node.key
ssl.certificate.location=/opt/miq/cvs/ssl/node.crt
ssl.ca.location=/opt/miq/cvs/ssl/ca.crt

I am using the following version of Kafka client:

github.com/confluentinc/confluent-kafka-go/[email protected]

However, the following scenarios do work:

1. When using IPv4:

bootstrap.servers=192.168.169.18:29092

# Enable the following properties for secure Kafka connection:
security.protocol=SSL
ssl.key.location=/opt/miq/cvs/ssl/node.key
ssl.certificate.location=/opt/miq/cvs/ssl/node.crt
ssl.ca.location=/opt/miq/cvs/ssl/ca.crt

2. When using the hostname:

2.1 Added the IPv6 entry in /etc/hosts:

cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

2001:db8:2:2000:169::18   kafkacluster05-server-1

2.2 Passed the hostname in bootstrap.servers:

bootstrap.servers=kafkacluster05-server-1:9092

# Enable the following properties for secure Kafka connection:
security.protocol=SSL
ssl.key.location=/opt/miq/cvs/ssl/node.key
ssl.certificate.location=/opt/miq/cvs/ssl/node.crt
ssl.ca.location=/opt/miq/cvs/ssl/ca.crt

3. When using ssl.endpoint.identification.algorithm=none with IPv6 in producer.properties:

bootstrap.servers=[2001:db8:2:2000:169::18]:9092

security.protocol=SSL
ssl.key.location=/opt/miq/cvs/ssl/node.key
ssl.certificate.location=/opt/miq/cvs/ssl/node.crt
ssl.ca.location=/opt/miq/cvs/ssl/ca.crt
ssl.endpoint.identification.algorithm=none

Request for Help:
We are looking for insights or recommendations to resolve the issue with IPv6 not working when using the IP address directly in bootstrap.servers.
Could there be any configuration changes or additional steps required to ensure IPv6 compatibility?

Thanks!