Setup MTLS using Cert-Manager as CA

[rizsk1220]

Hello Experts,
We are using Strimzi 0.46.0 with Kafka 4.0.0 and trying to setup MTLS between Kafka and Clients. We are using Cert-Manager to get the Certificates signed and are not using Strimzi's Internal CA. Below is Listener Configuration snippet for Kafka.yaml. The secret "combine-acme" contains Chain of Trust Certificates(root and issuing crts for Cert-Manager) and a Server Certificate which is Signed by Cert-Manager.

k

[scholzj]

A start would be to make sure everything is properly formatted and readable. You should also make sure you share the actual resources and not some templates.

[rizsk1220]

We are using Strimzi 0.46.0 with Kafka 4.0.0 and trying to setup MTLS between Kafka and Clients. We are using Cert-Manager to get the Certificates signed and are not using Strimzi's Internal CA. I have used kafkauser authentication as tls-external to look up for CN name as kafka user. Authentication is happening with this approach but authorization is failing.

i have a user as "capstrimziuser.cvs.com" and
certficate as
Certificate[1]:
Owner: CN=capstrimziuser.cvs.com, O="CVS Pharmacy, Inc.", L=Woonsocket, ST=Rhode Island, C=US

I used ssl.principal.mapping.rules: "RULE:^CN=([^,]+),.*$/$1/,DEFAULT" in the config at cluster level but seems broker is not updating this config and still erroring out as

[2025-08-11 16:35:47,112] ERROR Error when sending message to topic demo-topic with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [demo-topic]

[2025-08-11 16:40:47,112] WARN [Producer clientId=console-producer] The metadata response from the cluster reported a recoverable issue with correlation id 647819 : {demo-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)

[scholzj]

Are you sure you configure ssl.principal.mapping.rules correctly? Did you follow the docs?

[rizsk1220]

i have configured it under cluster config i.e.., spec.kafka.config
should i follow instruction mentioned in the below document to use custom
https://strimzi.io/docs/operators/latest/configuring.html

is there any other way to configure without using custom authentication.

[scholzj]

Yes, you can configure it only for custom authentication.

[rizsk1220]

will the bootstrap server configuration comes under listenerConfig, i believe instead of brokercertandchain we need to manually mount the ca.crt file as mentioned in the document.

bootstrap:
loadBalancerIP: {{ .Values.kafka.bootstrapIP }}
annotations:
cloud.google.com/load-balancer-type: "Internal"
cloud.google.com/internal-load-balancer-allow-global-access: "true"
cloud.google.com/internal-load-balancer-subnet: {{ .Values.kafka.subnet }}
brokers:
{{- range $index, $ip := .Values.kafka.externalIPs }}
- broker: {{ $index }}
loadBalancerIP: {{ $ip }}
annotations:
cloud.google.com/load-balancer-type: "Internal"
cloud.google.com/internal-load-balancer-allow-global-access: "true"
cloud.google.com/internal-load-balancer-subnet: {{ $.Values.kafka.subnet }}
{{- end }}

[scholzj]

You really have to start formatting the code and stop sharing any templates but the actual resources. Without that, it is useless.

[rizsk1220]

All right!
We have a Google Cloud load balancer assigned to each broker (including bootstrap). Since this is a custom setup, should I add this info in the listeners conf? I didn’t find any example in the docs for load balancing.

[scholzj]

should I add this info in the listeners conf?

Sorry, what exactly you mean with this info?

[rizsk1220]

I mean broker and bootstrap load balancing information.

[scholzj]

The type: custom is just authentication type. It does not change anything other in the listener configuration. You configure only the mTLS related stuff there - trusted certificates which signed the user certs, the mapping, etc.

[rizsk1220]

My requirement is to establish mutual authentication between the Kafka broker and the client using an external client CA. To achieve this, I disabled the Strimzi-generated client CA, created a dummy key, and provided the external client CA certificate. I used tls-external for the KafkaUser. However, because the User Operator matches the CN in the certificate instead of the KafkaUser name, topic authorization failed. I then attempted to use the custom authentication option to implement sl.principal.mapping.rules and mount the CA cert on listeners , but encountered the following error.

org.apache.kafka.common.errors.InvalidConfigurationException: Failed to load PEM SSL keystore /mnt/custom-external-9443-certs/ca.crt
Caused by: java.io.IOException: Unable to read file /mnt/custom-external-9443-certs/ca.crt

[scholzj]

You probably misconfigured something?

[rizsk1220]

Its working when i keep the user under -superUser.

Here are my observations

1. certificate example CN=capstrimziuser.cps.com, O="CPS, Inc.", L=Woonsocket, ST=Rhode Island, C=US
2. kafka user: capstrimziuser.cps.com
3. combination that is working
   authorization:
   type: simple
   superUsers:
   - capstrimziuser.cps.com

• name: external
  type: loadbalancer
  tls: true
  port: 9443
  authentication:
  type: custom
  sasl: false
  listenerConfig:
  ssl.client.auth: required
  ssl.principal.mapping.rules: RULE:^CN=([^,]+),.*$/$1/,DEFAULT

If i remove either username from the super user list or ssl.principal.ampping.rules. the acls are not working.

[rizsk1220]

It worked after changing the rule to RULE:^CN=([^,]+),.*$/CN=$1/,DEFAULT