Zookeeper anti-affinity is missing

[tgruenert]

Describe the bug
After a fresh install from example (3 instances) zookeeper run 3 times on the same node.

To Reproduce
Steps to reproduce the behavior:

1. install operator helm upgrade -i -f manual/blueprint/strimzi/operator-values.yaml strimzi strimzi/strimzi-kafka-operator
2. Create Custom Resource from examples/kafka/kafka-persistent.yaml , do some enhancements for connection via ingress
3. Run command kubectl apply -f kafka-persistent.yaml
4. See error

Expected behavior
zookeeper should install on every node one time

Environment (please complete the following information):

• Strimzi version: 0.28.0
• Installation method: Helm chart
• Kubernetes cluster: Kubernetes 1.20
• Infrastructure: Rancher on bare metal

kafka.yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.1.0
    replicas: 3
    listeners:
      - name: external
        port: 9094
        type: ingress
        tls: true
        authentication:
          type: tls
        configuration:
          bootstrap:
            host: kafka.example.org
          brokers:
          - broker: 0
            host: broker-0.kafka.example.org
          - broker: 1
            host: broker-1.kafka.example.org
          - broker: 2
            host: broker-2.kafka.example.org
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.1"
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 8Gi
        deleteClaim: false
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 8Gi
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

[scholzj]

Strimzi does not set any affinity by default. This is an intentional decision. You can configure affinity in the custom resource: https://strimzi.io/docs/operators/latest/full/configuring.html#affinity-str

[tgruenert]

Thank you @scholzj for your ultra fast response. The decision not to spread multiple broker across nodes seems unusual for me. But i am new to kafka and might not know enought about it. I did´nt thought to look for documentation on this. But your link is really useful and documentation is more than detailed to this. realy great.
Thank you!

[scholzj]

The decision is really simple - if you hardcode some affinity rules to enforce the pods to run on different nodes, you will most likely make it better for one group of users, make no difference for another and make it worse for the rest. For example

• Unless you set a hard (required) scheduling rules, they will be useless because they do not guarantee anything.
• If you set hard (required) scheduling rules by default, you will cause issues to many users using Strimzi for development or as part of their CI because they often do not have a fully HA cluster running across many nodes or zones.
• Also, what are the correct affinity rules according to you? Are they based on the pods being on different worker nodes? Or different zones? Or even regions? This is actually different from one user to another depending on their infrastructure and their environments.

So there is no universally correct setting for the affinity rules. And that is why our default is no affinity rules.

That does not mean that we are saying that it doesn't matter. If you are building a cluster where availability and reliability matters we would of course recommend your to distribute the pods. But it is basically up to you to set the right rules according to your environment and your requirements.

[tgruenert]

Thank you for your explanation. I can completely follow your arguments. Some other products take an 80/20 approach like elastic or mysql to meet most users requirements. If you define replicas >1 than they will be spread across available nodes. Every product can be modified individually. This is where i come from. And at strimzi the paradigm is different (for me!). No problem at all.