Migration of strimzi cluster from one K8s to another k8s

[senthil13]

Hi Experts,
Need your kind attention and suggestion on our issue.

Current setup
Currently we are running pretty old strimzi version (0.31.1) on our K8s cluster version v1.21.12

New setup

For various reasons, our container team is not going for an in-place upgrade, and they are building a new k8s cluster with Kubernetes version: v1.30.8.

Problem
Now I have to move the Kafka cluster with data from old k8s cluster to the new one.

Approach:
I am planning to deploy Strimzi's latest version. 0.45.0 (n-1 approach, which means one version below the latest), and going to use the MM2 to replicate the data since we need to preserve the offsets.

Now questions:

1. Does the above approach work? The source cluster is Zookeeper-maintained, and the new destination cluster will be Kraft-maintained.
2. Do I need to have the same version of Strimzi for the source and destination to migrate the data?
3. The source cluster metadata maintainer is ZooKeeper. Can I have Kraft mode for the destination cluster

If nothing works, then I have to create the same version of the Kafka Strimzi cluster at the destination with zookeeper and then migrate the data with MM2, then move the cluster to Kraft mode?

Thanks
Senthil

[scholzj]

For various reasons, our container team is not going for an in-place upgrade, and they are building a new k8s cluster with Kubernetes version: v1.30.8.

Are they not doing in-place upgrade only in this case because the Kubernetes version you use is so old? Or is it a general practice? Cluster migration using MM2 is a pretty nasty process. And if you would be expected to do it for every Kubernetes release, you would get crazy from it. So - to be honest - if that is the case, I would consider not using Strimzi/Kubernetes and moving to VMs. If it is one of thing, then obviously the migration and then continuing with in-place upgrades makes sense.

Approach: I am planning to deploy Strimzi's latest version. 0.45.0 (n-1 approach, which means one version below the latest), and going to use the MM2 to replicate the data since we need to preserve the offsets.

Honestly, I do not think the n-1 approach makes any sense. Historically, we do only very little patch releases. So, whatever bugs were in 0.46.0 will likely still be there once it becomes the n-1 release after we release 0.47.0.

0.46.0 also brings KAfka 4.0.0 with all kind of fixes and improvements to KRaft etc. So assuming you do not want to start with Strimzi 0.45 and ZooKeeper, 0.46 would be IMHO much better choice. It is now anyway few monts old I think.

1. Does the above approach work? The source cluster is Zookeeper-maintained, and the new destination cluster will be Kraft-maintained.

MM2 does not care about Zoo versus KRaft, so yes, you can do it.

2. Do I need to have the same version of Strimzi for the source and destination to migrate the data?

No, MM2 does not care what Strimzi version is used to run the Kafka cluster (or in fact whether it even is a Strimzi-based KAfka cluster). So you should be fine.

3. The source cluster metadata maintainer is ZooKeeper. Can I have Kraft mode for the destination cluster

Yes, you can have KRaft for the destination cluster.

[ppatierno]

+1 on what Jakub said about using the latest Strimzi 0.46.0 if your plan is using KRaft. The 0.45.0 comes from December, and despite we are still supporting it because of the KRaft migration feature, it's pretty old. We are working on a 0.45.1 to fix some CVEs but it 0.46.0 is way better considering Kafka 4.0.0 and its KRaft support.

[senthil13]

Thanks @scholzj and @ppatierno, I really appreciate your valuable comments.

Here is the outcome after discussing it with my peers.

1. We will be going to for 0.46.0, latest and greatest. Thanks for your insight comments. 👏
2. I am going to create the Kafka cluster with 0.46 and am going to use MM2 to migrate the data from the old version of the Strimzi Kafka cluster.
3. New source will be Kraft-managed; the old will be Zookeeper-managed, and it will be removed after migration.

Are they not doing in-place upgrade only in this case because the Kubernetes version you use is so old? Or is it a general practice? Cluster migration using MM2 is a pretty nasty process. And if you would be expected to do it for every Kubernetes release, you would get crazy from it. So - to be honest - if that is the case, I would consider not using Strimzi/Kubernetes and moving to VMs. If it is one of thing, then obviously the migration and then continuing with in-place upgrades makes sense.

The information I got from them is that the underlying OS version is old, so they are going to have a new cluster this time alone. It won't happen in the future.

Thanks
senthil