Cruise control behavior

[kspmmitt]

Hello, having few doubts about behavior of cruise control as below, please give inputs.

1. Kafka rebalance custom resource created in cruise control automated approval flavor has always optimization proposal mode as “full” even if given customized values like add-brokers/remove-brokers. Is this right behavior?
2. For automated cruise control, Kafka rebalance custom resource is created automatically but for manual it must be created by user so what if user flips between automated and manual. Also, can outside KR resource be used in automated mode?
3. Sometimes Kafka rebalance resource vanishes, not sure about this behavior.
4. Does cruise control rebalances its internal managed topics as well like - strimzi.cruisecontrol.partitionmetricsamples, strimzi.cruisecontrol.modeltrainingsamples?

[ppatierno]

Kafka rebalance custom resource created in cruise control automated approval flavor has always optimization proposal mode as “full” even if given customized values like add-brokers/remove-brokers. Is this right behavior?

Not sure what you mean by "automated approval flavor", maybe when you create a KafkaRebalance custom resource by using the strimzi.io/rebalance-auto-approval: "true" annotation? If yes, it's independent of the mode. You can even create an auto-approval enabled KafkaRebalance for add-brokers/remove-brokers. The goal of this feature is that when the KafkaRebalance moves to ProposalReady state it's approved automatically and moves to Rebalancing.

For automated cruise control, Kafka rebalance custom resource is created automatically but for manual it must be created by user so what if user flips between automated and manual. Also, can outside KR resource be used in automated mode?

Even if I am not sure what you mean by "automated cruise control" even because there isn't right now such a feature in Strimzi. The only automation is with auto-rebalancing which applies when you scale up/down your cluster but not during the normal cluster life where you need to create your KafkaRebalance resources by yourself for manual rebalancing.

Sometimes Kafka rebalance resource vanishes, not sure about this behavior.

If a KafkaRebalance is manually created by the user than it won't be deleted at the end, it will just move to Ready state. If you are talking about the KafkaRebalance automatically created by the operator when running an auto-rebalancing on scale up/down, then yes. They are created by the operator and deleted by the operator itself when the rebalancing is done.

Does cruise control rebalances its internal managed topics as well like - strimzi.cruisecontrol.partitionmetricsamples, strimzi.cruisecontrol.modeltrainingsamples?

Yes AFAIK. They are normal topics.

[ppatierno]

I was asking that if cruise control optimization mode is changed from "full" to "add-brokers" say then still KafkaRebalance resource is created with "full" mode only and name of resource also has "full" string. Ler me know please else may be something wrong done at may machine?

Tbh I don't get what you mean. Are you talking about a KafkaRebalance created by you manually? Or the auto-rebalancing on scaling up/down?

Also, if any new broker is added or removed then cruise control pod gets re-started but is this behavior same for other scenarios like preferred leader election etc.?

When the cluster topology changes, CC pod is restarted because it needs to be updated with some info about the new topology. It doesn't happen in other cases.

[kspmmitt]

Can you please give info. for preferred leader election scenario above, is it done by cruise control, how to check it?

[ppatierno]

There is a dedicated PreferredLeaderElectionGoal goal in CC for it. You can configure it in your KafkaRebalance. How to do it ... it's in the doc.

[kspmmitt]

Yes, I know and tried the same. Leader also changed but no logs/info. in cruise control so how to verify that it is done by cruise control or?

[kspmmitt]

Hello,
Can you please help to know about rebalancing failure and success impact w.r.t existing kafka producers/consumers traffic?
It will be gr8 if any documentation around this?
Rgds,
Kamal