librdkafka 1.6.0 , C++ API producer, producing immediately after topic creation SOMETIMES result in errors (in RdKafka::Producer::produce call) #3977
aKumara123
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
librdkafka version => 1.6.0
topic.metadata.propagation.max.ms=60000
Topic auto creation NOT used (in our use case different topics need to have different partition counts). OS = Red-hat Linux 7.9
Message is sent(RdKafka::Producer::produce) immediately after successfully creating the topic (rd_kafka_CreateTopics => rd_kafka_queue_poll => if NOT timed out, check errors with rd_kafka_event_error and rd_kafka_event_error+rd_kafka_CreateTopics_result_topics+rd_kafka_topic_result_error => RD_KAFKA_RESP_ERR_NO_ERROR is taken as success)
Same RdKafka::Producer used to create topics and Produce. Both calls made in in same thread.
But the produce call/delivery fails some times with below errors(other times it is successful),
Produce call fail : ERR__UNKNOWN_TOPIC, ERR__FATAL
Delivery fail (Callback) : ERR__DESTROY, ERR__UNKNOWN_PARTITION
What is the recommended solution here ?
Add a sleep after each topic creation ?
OR
any other ?
[same question posted on Stack overflow too, but no response received yet 2022/09/13]
Beta Was this translation helpful? Give feedback.
All reactions