fetch.wait.max.ms setting isn't respected

[dmariassy]

Hi 👋

First of: thank you for the great library!

App

I'm running some tests using the library's ruby binding using a no-op consumer, where the only "processing" I do per record, is to emit a Datadog metric to measure message latency, calculated as the delta between current wall time and message CreateTime.

Configuration

I'm using the following configs:

fetch.wait.max.ms: 200,
fetch.min.bytes: 100000000, # Max allowed value
max.partition.fetch.bytes: 1000000000, # Max allowed value
receive.message.max.bytes: 2147483647, # Max allowed value
fetch.max.bytes: 2147483135 # receive.message.max.bytes - 512

Expectation

Based on the above configs, I would expect that latency would be greater than or equal to 200 ms, since all the other options that might short-circuit a fetch request (fetch.min.bytes, fetch.max.bytes, etc.) are effectively disabled.

Results

Latency is consistently below 100 ms. Obviously, lower latencies are a good thing in general 🙂 However, I want to make sure that my mental model of how librdkafka implements the fetch protocol is correct so I can assist our users with misbehaving apps. These results simply don't make sense to me.

Am I missing something?

TIA 🙏

[edenhill]

We don't do anything with fetch.wait.max.ms in librdkafka other than pass it to the broker in the FetchRequest, so I recommend setting debug=protocol and looking at the RTT of FetchResponses to see how long it is actually blocking on the broker.

[dmariassy]

Thanks, I'll give that a try!