Number of broker threads created

[masenov9607]

Docs states that Librdkafka creates 1 main thread and one thread per broker, for each client instance.

In the implementation of rd_kafka_new
it is created additional internal broker thread which is not connected to any broker and serves for handling
ops which are not related to the brokers thread. So there are 2 + NUM_BROKERS, not 1 + NUM_BROKERS .

      rd_kafka_broker_add(rk, RD_KAFKA_INTERNAL, RD_KAFKA_PROTO_PLAINTEXT,
                                "", 0, RD_KAFKA_NODEID_UA);

Is it possible that amount to be reduced to 1 + NUM_BROKERS?

[edenhill]

There's currently no trivial way to reduce the numbers of threads in librdkafka, it would require a large design overhaul.

Threads are, however, quite cheap on most systems. What's the problem you are seeing with many threads?

[chenhongluo]

@edenhill It looks like rdkafka never releases broker threads at runtime. This looks a little stange.

[chenhongluo]

Even if the server has a node permanently offline, the client still does not release the thread.