Kafka Exporter resolving to latest supported Kafka version

[marceloavan]

Hey,

I was wondering about why the images map for helm consider that the kafka exporter and cruise control evaluates to the latest Kafka supported version instead of keeping it related with the version requested on the CR:

https://github.com/strimzi/strimzi-kafka-operator/blob/main/packaging/helm-charts/helm3/strimzi-kafka-operator/templates/_kafka_image_map.tpl#L9

I realized this because I have to publish the images on the company's registry to be used, and I was prepared for 3.9.0.

I mean, not a big issue, I was able customized directly the exporter image, but still a doubt and perhaps an enhancement.

[im-konge]

Hey, I'm not Helm expert and I don't remember the details.
But I guess that it's because those fields are really just for the image directly. Those fields are type of String and not Maps as the Kafka images for example. And because it's generated beforehand, the latest Kafka version (and its image) is always picked. This mirrors the YAML file for installation using YAML manifests - as you can see here https://github.com/strimzi/strimzi-kafka-operator/blob/main/packaging/install/cluster-operator/060-Deployment-strimzi-cluster-operator.yaml#L51.

Those are anyway env variables for the operator.

In case that you want to configure those, you can configure them in the values.yaml to exact image with desired Kafka version you need.

[scholzj]

Yeah. There is no mapping. It uses the image configured in the environment variable. It should be possible to change it even in the Helm Chart if needed (All Kafka images have the same Kafka Exporter version). The same applies also for Cruise Control.

[marceloavan]

I believe I got the idea around environment variables, in the end that templates I shared will result in a similar deployment configuration to @im-konge sharing.

But the gap is still the "why" there is this difference comparing entity operator with kafka exporter (and CC). I mean, the behavior I got when deploying a Kafka CR targeting 3.9 was:

• connect and mm2 version suffixed with -kafka-3.9.0 (originally mentioned entity operator by mistake)
• kafka exporter version suffixed with -kafka-4.0.0

As result, in my case with a custom docker repository, where I have just the 3.9 images, my approach was set the specific image in the kafkaExporter configuration directly on the Kafka CR.

[scholzj]

The Entity Operator has nothing to do with the Kafka image. It does not use any of them. It uses the Operator image.

The Kafka Exporter and Cruise Control have also - strictly speaking - nothing to do with the Kafka version. They just piggyback the latest Kafka version container image. Ideally they would have their own container images. But due to resource and time constraints, they share the image with Kafka.

In any case, I would always recommend you to mirror all the images from the Strimzi release as you never know when someone might need one of them - either intentionally or by mistake - and run into issues.

[marceloavan]

The Entity Operator has nothing to do with the Kafka image.

You are right, my bad, my brain tricked me on this. The thing is about the resources sharing the Kafka image: CC and exporter resolves to the latest supported Kafka version, while connect, mm2 and kafka itself resolves to the defined version on the CR. So replace the entity operator in my example by any of these haha

I would always recommend you to mirror all the images from the Strimzi release as you never know when someone might need one of them - either intentionally or by mistake - and run into issues

Yeah, seems to be the best. Will do.

Thanks for the quick support, guys!

[scholzj]

The thing is about the resources sharing the Kafka image: CC and exporter resolves to the latest supported Kafka version, while connect, mm2 and kafka itself resolves to the defined version on the CR. So replace the entity operator in my example by any of these haha

Yeah, I know it could be a little bit confusing. But the KAfka, Connect and MM2 images need to use different versions because they are versioned. Cruise Control and Kafka Exporter are not versioned - the same version is used for all Kafka versions. That is why they are configured separately and with separate variables. Ideally, one day we should try to get back to using separate images as well - or maybe just use ImageVolumes for it etc.