Strimzi topic operator creating automatic topics

[shreyasarani23]

Following is the yaml file for creating topics

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: ad-click
  labels:
    strimzi.io/cluster: my-kafka-cluster
spec:
  topicName: ad_click
  partitions: 3
  replicas: 3
  config:
    min.insync.replicas: 2

I want my topic to be named as ad_click. But I know that kubernetes doesn't support metadata.name to have underscore. So I have used spec.topicName and created the topic. But when I do kubectl get KafkaTopics -n kafka I see that the Topic Operator has created topics by appending a hash to the topic name. Even though I have used spec.topicName why it is creating the topics appending hash to the topic? And also these are not in ready state.

please help me to resolve this issue.

kubectl get KafkaTopics -n kafka
NAME                                                                                               CLUSTER            PARTITIONS   REPLICATION FACTOR   READY
ad-click                                                                                           my-kafka-cluster   3            3                    True
ad-click---8d8d553a97f8d2c936d8069b16346eb58e209634                                                my-kafka-cluster   3            3
add-reminder-icon                                                                                  my-kafka-cluster   2            3                    True
add-reminder-icon---a5517286a3495ccd16c5b588577ee0a64f666871                                       my-kafka-cluster   2            3
app-exit                                                                                           my-kafka-cluster   3            3                    True
app-exit---28aefee7fbc40e162373c31d2f6559fdacffa030                                                my-kafka-cluster   3            3

kubectl describe KafkaTopic ad-click---8d8d553a97f8d2c936d8069b16346eb58e209634  -n kafka
Name:         ad-click---8d8d553a97f8d2c936d8069b16346eb58e209634
Namespace:    kafka
Labels:       strimzi.io/cluster=my-kafka-cluster
Annotations:  <none>
API Version:  kafka.strimzi.io/v1beta2
Kind:         KafkaTopic
Metadata:
  Creation Timestamp:  2021-07-13T17:50:48Z
  Generation:          1
  Managed Fields:
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
    Manager:         okhttp
    Operation:       Update
    Time:            2021-07-13T17:51:04Z
  Resource Version:  19939221
  Self Link:         /apis/kafka.strimzi.io/v1beta2/namespaces/kafka/kafkatopics/ad-click---8d8d553a97f8d2c936d8069b16346eb58e209634
  UID:               d8394fc8-c335-4b70-b114-cc6c82a4bbc1
Spec:
  Config:
    min.insync.replicas:  2
  Partitions:             3
  Replicas:               3
  Topic Name:             ad_click
Status:
  Conditions:
    Last Transition Time:  2021-07-13T17:51:03.987012Z
    Message:               The state store, topic-store, may have migrated to another instance.
    Reason:                InvalidStateStoreException
    Status:                True
    Type:                  NotReady
  Observed Generation:     1
  Topic Name:              ad_click
Events:                    <none>

[sknot-rh]

What version of Strimzi do you use? I tried your reproducer with the latest image. Everything works as expected and the Topic resource is ready. Also listing topics from the Kafka broker contains the correctly named topic.

[sknot@localhost strimzi-kafka-operator]$ oc get kt 
NAME                                                                                               CLUSTER      PARTITIONS   REPLICATION FACTOR   READY
ad-click                                                                                           my-cluster   3            3                    True
consumer-offsets---84e7a678d08f4bd226872e5cdd4eb527fadc1c6a                                        my-cluster   50           3                    True
strimzi-store-topic---effb8e3e057afce1ecf67c3f5d8e4e3ff177fc55                                     my-cluster   1            3                    True
strimzi-topic-operator-kstreams-topic-store-changelog---b75e702040b99be8a9263134de3507fc0cc4017b   my-cluster   1            1                    True

[sknot@localhost tmp]$ oc exec -ti my-cluster-kafka-0 -- bash bin/kafka-topics.sh --list --bootstrap-server localhost:9092
__consumer_offsets
__strimzi-topic-operator-kstreams-topic-store-changelog
__strimzi_store_topic
ad_click

[shreyasarani23]

sorry I didn't check properly I have the internal topics

kubectl get KafkaTopics -n kafka | grep consumer
consumer-offsets---84e7a678d08f4bd226872e5cdd4eb527fadc1c6a                                        my-kafka-cluster   50           3                    True

 kubectl get KafkaTopics -n kafka | grep strimzi
strimzi-store-topic---effb8e3e057afce1ecf67c3f5d8e4e3ff177fc55                                     my-kafka-cluster   1            3                    True
strimzi-topic-operator-kstreams-topic-store-changelog---b75e702040b99be8a9263134de3507fc0cc4017b   my-kafka-cluster   1            1                    True

[shreyasarani23]

So what can I do now is there any solution ?
I have around 70-75 topics deleting each topic associated with hash manually is a tedious task.
Any other work around?

[sknot-rh]

I do not tnink there is any workaround :( if the hsh-suffixed topics are created, that means the mapping between hashed-topic->kube resource already exists in strimzi-store-topic.
Maybe this could work. But I am not 100% sure it does the right thing. @alesj can you help there?

• stopping TO
• removing all k8s Kafka Topic resources
• removing strimzi-store-topic from the kafka
• creating all your KafkaTopic resources
• starting TO

[shreyasarani23]

Hi just an update I was able to delete the topics associated with hash with the following command

kubectl get KafkaTopic -n kafka | grep -v True | awk '{print $1}' | xargs kubectl -n kafka delete KafkaTopic

-v -> --invert-match select non-matching lines
awk '{print $1}' -> prints only 1st column i.e. topic name appended with hash
xargs-> passes the output of awk command to the kubectl command.

All the topics will be deleted one by one. But still not clear why these were created,

[sknot-rh]

I am glad you fixed your cluster. As jakub suggested the full log from the time the issue occurred would help us to understand what exactly happened. My guess is that TO recreated the k8s KafkaTopics from topics present in the brokers before they were created by you.