docs(config): adds kraft-related annotations to kraft config examples

[PaulRMellor]

Documentation

Updates the Kafka config examples in the docs with the Kraft-related annotations

Checklist

Please go through this checklist and make sure all applicable tasks have been done

• Write tests
• Make sure all tests pass
• Update documentation
• Check RBAC rights for Kubernetes / OpenShift roles
• Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
• Reference relevant issue(s) and close them after merging
• Update CHANGELOG.md
• Supply screenshots for visual changes, such as Grafana dashboards

[scholzj]

@strimzi/maintainers Should we instead consider removing the annotations as part of 0.48.0?

This was always planned at some point, but we never set a fixed date. But in general:

• The protection for screwing up existing ZooKeeper-based clusters that were no migrated is done based on the status of the Kafka CR. So not checking the annotations might impact only some old Kafka clusters (likely managed by Strimzi 0.39 or older?)
• If someone tries to deploy a new ZooKeeper-based cluster, it should fail during the deployment due to missing node pools / controller node pools. So while it would fail a little bit later in the reconciliation, I do not think it would be a major problem as used can either just fix it or delete the old cluster and deploy a new one with Kraft.

I'm happy to have this merged if others think that we should keep the annotations for the time being. But we should probably at least make some ballpark agreement when do we plan to remove them.

[im-konge]

@strimzi/maintainers Should we instead consider removing the annotations as part of 0.48.0?

This was always planned at some point, but we never set a fixed date. But in general:

• The protection for screwing up existing ZooKeeper-based clusters that were no migrated is done based on the status of the Kafka CR. So not checking the annotations might impact only some old Kafka clusters (likely managed by Strimzi 0.39 or older?)
• If someone tries to deploy a new ZooKeeper-based cluster, it should fail during the deployment due to missing node pools / controller node pools. So while it would fail a little bit later in the reconciliation, I do not think it would be a major problem as used can either just fix it or delete the old cluster and deploy a new one with Kraft.

I'm happy to have this merged if others think that we should keep the annotations for the time being. But we should probably at least make some ballpark agreement when do we plan to remove them.

In my opinion, now when the ZK support is removed and we have only KRaft, I would remove the annotations and have them enabled by default. I think it would make others' lives a bit easier as well. Because you cannot anyway deploy new clusters without NodePools, so I would remove them.

[ppatierno]

+1 from me as well.
Since 0.46.0, only KRaft clusters are supported so the strimzi.io/kraft annotation is something that should be by default.
The same applies for strimzi.io/node-pools which are now the only way to deploy the cluster implied by having only KRaft supported (so you need node pools).

[katheris]

+1 from me for removing the annotations in 0.48.0.

For this PR specifically, I'm potentially missing some context, but do we need the annotations on every single Kafka snippet? I understand they are required but for example say in this one for explaining how to configure the rack it omits most fields, so why not also omit the annotations?

apiVersion: {KafkaApiVersion}
kind: Kafka
metadata:
  name: my-cluster
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafka:
    # ...
    rack:
      topologyKey: topology.kubernetes.io/zone
    # ...
    ```

[PaulRMellor]

Removing sounds good to me.
Depending on the timeline, this can be closed or merged

[scholzj]

As this was mostly my idea, I opened #11686 to remove the annotations. Assuming it is approved and merged, we should be able to just close this PR.

[katheris]

Closing since we are handling this by removing the annotations