Getting kafka cli to work!

[mokhos]

Hi, so as the title states, I'm trying to get the cli to work and I just tried every config I found but no luck to this moment.

This is my configuration based on what I found here:

The kafka cluster:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: test-cluster 
  namespace: new-kafka 
spec:
  kafka:
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
            type: tls
        configuration:
          brokerCertChainAndKey:
            certificate: tls.crt
            key: tls.key
            secretName: new-kafka-tls
      - name: external
        port: 9094
        type: ingress
        tls: true
        authentication:
            type: scram-sha-512
        configuration:
...
...
    config:
...
    authorization:
      type: simple
      superUsers:
        - CN=admin

and the super user:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: admin
  namespace: new-kafka
  labels:
    strimzi.io/cluster: test-cluster
spec:
  authentication:
    type: tls

and the config.properties:

bootstrap.servers=test-cluster-kafka-bootstrap.new-kafka.svc.cluster.local:9092
sasl.mechanism=SCRAM-SHA-512
security.protocol=SASL_PLAINTEXT
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username='admin' password='x';

The command I'm running:

./kafka-topics.sh --bootstrap-server test-cluster-kafka-bootstrap.new-kafka.svc.cluster.local:9092 --topic "test" --create --command-config /tmp/config.properties

with port setting to 9092, I get "authentication failed"!
with port setting to 9093, I get a different error.

I'm really confused. I appreciate if anyone can help here.

Thanks in advance

[scholzj]

Well, you create the user with tls authentication. So you cannot use it for SCRAM-SHA authentication. No idea where you even got the password. You have to change the authentication type for the user and then use the credentials you get for it.

[mokhos]

So after changing the configuration to scram-sha-512, this is the error I'm getting:

2024-03-14 14:57:19,045 INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.253.4.215 (channelId=10.253.4.215:9092-10.253.4.215:50744-26) (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-0-ListenerName(PLAIN-9092)-SASL_PLAINTEXT-6]

again the kafka setting:

kind: Kafka
metadata:
  name: test-cluster 
  namespace: new-kafka 
spec:
  kafka:
    version: 3.6.1
    replicas: 3 
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        authentication:
            type: scram-sha-512

the config.properties this time looks like this:

bootstrap.servers=test-cluster-kafka-bootstrap.new-kafka.svc.cluster.local:9092
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="ef5ygWFbJmxSGt9xdQPaJmonmbvyYTLl";

What am I missing here? ://

[scholzj]

The security.protocol?

[mokhos]

I added the security.protocol=SASL_PLAINTEXT to my config.properties and this is the new error:

[kafka@test-cluster-kafka-0 bin]$ ./kafka-topics.sh --bootstrap-server test-cluster-kafka-bootstrap.new-kafka.svc.cluster.local:9092 --topic "y-topic" --create --command-config /tmp/config.properties Error while executing topic command : Authorization failed. [2024-03-14 15:16:16,065] ERROR org.apache.kafka.common.errors.TopicAuthorizationException: Authorization failed.

I'm really confused!

[scholzj]

That means that you authenticated I guess, but are missing some ACL rights. If you use SCRAM-SHA, the user name will be just admin. So you should use this I guess:

    authorization:
      type: simple
      superUsers:
        - admin

[mokhos]

Thank you so much Jakub. It is finally working