Why is tokio multi threaded runtime so much slower than current thread?

[bennetthardwick]

I was working on a project that downloads large files with reqwest when I noticed that the multi threaded runtime was downloading files significantly slower than the current thread runtime. I understand that there is going to be some extra overhead with multiple threads - but how significant a drop in performance would be reasonable?

I've written some simple benchmarks and put the code up here: https://github.com/bennetthardwick/reqwest-benches.

With a local file server that serves a 16 GiB/s I get the following speeds:

bench	speed
reqwest_current_thread	3276 MiB/s
reqwest_multi_thread	680 MiB/s
go	2804 MiB/s

The code looks like this with the only difference between the two being flavor = "current_thread" set for one.

#[tokio::main]
async fn main() {
    println!("Starting reqwest_multi_thread...");

    let mut stream = reqwest::get("http://localhost:3000")
        .await
        .unwrap()
        .bytes_stream();

    let mut byte_count: usize = 0;
    let start = Instant::now();

    while let Some(next) = stream.next().await {
        let bytes = next.unwrap();
        byte_count += bytes.len();
        black_box(bytes);
    }

    // server will return 16GiB of ones
    assert_eq!(byte_count, 16 * 1024 * 1024 * 1024);

    println!(
        "reqwest_multi_thread ran at an average of {} MiB/s",
        (byte_count as f64) / 1024. / 1024. / start.elapsed().as_secs() as f64
    );
}

Is this kind of slowdown to be expected or am I doing something wrong somewhere? I would appreciate any help.

[Darksonn]

The main thread is not a worker thread when using the multi-thread runtime. Wrap the body in tokio::spawn to actually execute the code on a worker thread.

[mox692]

If you spawn a task on a non-worker thread, the task is first added to the global queue. Then, one of the worker threads tries to pop the task from the global queue and store it in the local queue, which introduces extra overhead compared to spawning the task directly on a worker thread.

is this documented somewhere?

Probably not, as far as I know.

[barafael]

Is this even possible to change? I'm a little bit surprised by this fact but it does make sense.

Where is the best point to document this?

This kind of incentivizes to have

#[tokio::main] 
async fn main() {
    ...
    let handle = tokio::spawn(async move {
        loop {
            let ... = listener.accept().await;
            tokio::spawn(...)
       }
    };
    handle.await.unwrap();
}

which means to think about application shutdown, so that's good. But it should be noted somewhere, that IIUC, "awaiting in main is inherently slower than awaiting in a tokio task".

[mox692]

There was a discussion about making the block_on thread a worker thread (#5446), but this has not been implemented.

Where would be the best place to document this?

Hmm, it's a bit tricky, but the runtime module docs or the block_on docs come to mind.

[mox692]

Oh, actually there is already documentation that mentions this:

• https://docs.rs/tokio/latest/tokio/attr.main.html#non-worker-async-function
• https://docs.rs/tokio/latest/tokio/runtime/struct.Runtime.html#non-worker-future

But I guess we could refine them if it's still unclear.

[barafael]

Very good, I didn't know these docs. Probably good enough