Running axum in a blocking dedicated thread

[danbachar]

Summary

I want to run axum on a blocking dedicated thread using tokio's task::spawn_blocking, and I have other async tasks which I run with spawn. I want the tasks spawned using the normal spawn to run on a thread which is not the same thread as the axum http task, because the http task is IO-heavy and executes a number of syscalls. Before I refactored the code to spawn the http task as blocking, the http task would sometimes be scheduled on the same thread as the other tasks, which would cause them to hang until the http task is done answering an HTTP request.

My problem is that currently the HTTP task does not respond to HTTP requests (http requests hang).
Does anyone have a clue why it's happening?
My setup is as following:

// spawn a thread to process http requests
let http_task = {
    tokio::task::spawn_blocking(move || {
        axum::serve(listener, create_routes().with_state(multithread_config))
    })
};

// spawn a task to receive new stuff
let receive_new_stuff_task = {
    tokio::spawn(async move {
        loop {
            let something = new_receiver.recv().await.unwrap();
            // do something with something
        }
    })
};

// spawn a task to receive delete requests for things
let delete_stuff_task = {
    tokio::spawn(async move {
        loop {
            let id = delete_receiver.recv().await.unwrap();
            // delete thing with id
        }
    })
};

_ = terminate_receiver.recv().await;
http_task.abort();
delete_stuff_task.abort();
receive_new_stuff_task.abort();

axum version

0.7.9

[jplatte]

If you're not .awaiting axum::serve, it does nothing. It's no surprise that the code you shared doesn't work.

If some parts of your HTTP service issue blocking syscalls, the solution is to move those parts out to be executed in blocking threads, not move the entire HTTP service into a blocking thread.

[danbachar]

hey @jplatte, thanks for answering my question so fast!
I also had the thought of spawning tasks for the individual http routes to separate threads, but was worried that when I do that runtime wouldn't await for the results of those threads, yielding empty http responses. I guess I'd also have to wrap the state my http routes need in an arc?

I'm putting below an example path that I use in the app:

async fn get_handler(
    State(multithread_config): State<MultithreadConfig>,
    Path(id): Path<String>,
) -> ValueOrError { // ValueOrError is an axum wrapper around Result that implements IntoResponse
    let result = do_something(); // do_something is a long-running IO-bound call

    ValueOrError(result)
}

[jplatte]

When you call spawn_blocking, you get back a JoinHandle that you can .await to wait for the blocking work to finish, and receive the result. That is, you should be able to replace your let result line from the example with this:

let result = tokio::task::spawn_blocking(|| do_something())
    .await
    .expect("blocking work panicked");

You don't have to wrap your state in an Arc for this, though that is often a good idea regardless of whether handlers call blocking tasks, as axum clones the state a lot.