Not authorized to access topics

[houas6]

Kafka Authorization Issue (mTLS + OAuth2) - Topic Authorization Failed

Hello everyone,

I'm trying to implement mTLS and OAuth2 authentication for Kafka using Strimzi on AKS, but I'm encountering the following error when attempting to consume/produce from my topic my-topic:

ERROR [Consumer clientId=console-consumer, groupId=console-consumer-68945] Topic authorization failed for topics [my-topic] (org.apache.kafka.clients.Metadata)
[2025-03-28 03:32:01,950] ERROR Error processing message, terminating consumer process: (org.apache.kafka.tools.consumer.ConsoleConsumer)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [my-topic]

Kafka Cluster Configuration

Here is my Kafka cluster configuration using Strimzi:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: mycluster
  namespace: kafka
  labels:
    app: mycluster
  annotations:
    cert-manager.io/cluster-issuer: "****"
spec:
  kafka:
    replicas: 3
    storage:
      type: jbod
      volumes:
        - id: 0
          type: persistent-claim
          size: 50Gi
          deleteClaim: true
    config:
      auto.create.topics.enable: "false"
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      ssl.client.auth: "required"
    authorization:
      type: simple
      superUsers:
        - CN=my-user
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: external
        port: 9094
        type: ingress
        tls: true
        authentication:
          type: oauth
          validIssuerUri: "https://sts.windows.net/************/"
          jwksEndpointUri: "https://login.microsoftonline.com/**************/discovery/v2.0/keys"
          checkAccessTokenType: false
          clientId: "aacae035-ad39-4000-9db0-5fd6dacd548d"
          clientSecret:
            secretName: kafka-oauth-secret
            key: secret
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 20Gi
      deleteClaim: true
  entityOperator:
    topicOperator: {}
    userOperator: {}

Kafka User Configuration

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: my-user
  namespace: kafka
  labels:
    strimzi.io/cluster: mycluster
spec:
  authentication:
    type: tls
  authorization:
    type: simple
    acls:
      # Consumer ACLs for my-topic
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operations:
          - Describe
          - Read
        host: ""
      - resource:
          type: group
          name: my-topic
          patternType: literal
        operations:
          - Read
        host: ""
      # Producer ACLs for my-topic
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operations:
          - Create
          - Describe
          - Write
        host: "*"

Any help would be greatly appreciated! 🙏

[see-quick]

Can you please share full logs (i.e., operators and Kafka + clients)? Also configurations of the clients would be really helpful. Thanks.

You are talking about implementing mTLS and OAuth2 authentication for Kafka using Strimzi on AKS, but you also configure authorization, so I'm not sure what your goal is.

[cmtly]

The group ACL grants read access for a group name of my-topic, however, the logs show a client connecting with a clientId of console-consumer. Seems to suggest a simple consumer group name mismatch.

[Frawless]

client.id is not the same thing as group.id, you should check if group.id=my-topic in your consumer config, if not, change config either in your consumer or in ACL.