-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Kafka Message loss when Kafkaserver goes down #4099
Replies: 1 comment · 6 replies
-
Is there an error associated with that delivery result? There should be |
Beta Was this translation helpful? Give feedback.
All reactions
-
Then the message is not lost, it failed to deliver because the topic or partition doesn't/no-longer exists - if this happens when a broker goes down it is typically because the topic is not properly replicated. Use a replication factor of 3. |
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Thanks for quick reply @edenhill. I agree with you on replication factor to be 3, But for some reason we are using single kafka server and Zookeeper. How do we overcome with this message loss. I understand that by having Not sure for what reason message is not able to deliver. Is there any setting is available for delivery timeout. kindly give a thought on this. Below is the another message on the same scenario, sometimes Exception message says And also I observed that Kafka server is up after 1 minute then the message is lost. if kafka is up before 1 min. then message is able to produce it. |
Beta Was this translation helpful? Give feedback.
All reactions
-
@edenhill Appreciate your inputs to understand the above behaviour further |
Beta Was this translation helpful? Give feedback.
All reactions
-
@edenhill and @mhowlett ` ` |
Beta Was this translation helpful? Give feedback.
All reactions
-
Could you please provide thoughts on this. Appreciate your time on this. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I have a scenario that when Kafka server or Zookeeper both goes down, but the producer application trying to send a message to Kafka broker.
When the Kafka broker or Zookeeper comes online, messages is getting lost.
Below are my configuration to reproduce
I have a scenario that when Kafka server or Zookeeper both goes down, but the producer application trying to send a message to Kafka broker.
When the Kafka broker or Zookeeper comes online, messages is getting lost and i see the producer delivery result offset has Offset: -1001 and Partition [Any]
I used the above mentioned setting to produce the message using Confluent kafka nuget package version 1.8.2 from .Net code.
Appreciate your inputs in advance @edenhill
Beta Was this translation helpful? Give feedback.
All reactions