[unbounded_channel] Q: How to refuse block recycling in reclaim_block ?

[gagbo]

Hello,

First of all, thanks to all the people contributing to tokio, it's been nice to use :)

I have a question about the behaviour of mpsc::unbounded_channel and how to possibly customize it.

Context

I am using tokio in long-running daemons, where I use message passing between a single sender and a single receiver through an unbounded channel. I noticed that the memory seemed to be leaking and I am trying to understand why this may be.

As the queue seems to be relatively small most of the time (maybe hundreds of MB) with occasional bursts, I wonder why I don't see the resident memory used by my process go down after the big bursts and back to a reasonable queue size.

Question

If I understand correctly, this snippet (reproduced below) tries to recycle the freed memory in the unbounded channel when the receiver takes a message off the queue, am I correct ?

tokio/tokio/src/sync/mpsc/list.rs

Lines 204 to 215 in 03969cd

	// TODO: Unify this logic with Block::grow
	for _ in 0..3 {
	match curr.as_ref().try_push(&mut block, AcqRel, Acquire) {
	Ok(_) => {
	reused = true;
	break;
	}
	Err(next) => {
	curr = next;
	}
	}
	}

If so, is it possible to somehow (through a setting in the constructor or an API I could use downstream) to "force reclaim" the blocks to actually get memory back, or to skip this recycling loop ?

[Darksonn]

There isn't currently any way to disable block recycling.

Regardless, have you verified that it's not just your memory allocator holding on to the memory rather than it actually still being owned by the channel?

[gagbo]

I haven't really looked much more into it than what I exposed in the first post. I do think that there are some other issues in my code, it's just that a "quick fix" for now seemed to be the ability to at least disable block recycling and/or force a big memory reclaim once in a while.

I don't think I'm using any specific allocator, so I didn't think to dig more into this direction, I'll try to see for sure.

Thanks for the clarification :)