Problem with rate limit (axum 0.7.4)

[dayvejones]

Summary

Hello!

I found a recommendation from David on using the Rate limiting (tower) - #987. But it doesn't work for me for all routes.

[package]
name = "axum-rate-limitting"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.7.4"
tokio = { version = "1.35.1", features = ["macros", "rt-multi-thread"] }
tower = { version = "0.4", features = ["full"] }

use std::time::Duration;
use axum::{routing::get, Router, error_handling::HandleErrorLayer, BoxError, http::StatusCode};
use tower::{buffer::BufferLayer, limit::RateLimitLayer, ServiceBuilder};

#[tokio::main]
async fn main() {
    let app = Router::new()
        .route("/1", get(handler1))
        .route("/2", get(handler2))
        .layer(
            ServiceBuilder::new()
            .layer(HandleErrorLayer::new(handle_too_many_requests))
            .layer(BufferLayer::new(1024))
            .layer(RateLimitLayer::new(5, Duration::from_secs(100)))
        )
        .route("/3", get(handler3));

    let listener = tokio::net::TcpListener::bind("127.0.0.1:3000")
        .await
        .unwrap();

    axum::serve(listener, app).await.unwrap();
}

async fn handler1() -> Result<String, StatusCode> {
    //tokio::time::sleep(Duration::from_secs(3)).await;
    Ok("handler1".to_owned())
}

async fn handler2() -> Result<String, StatusCode> {
    Ok("handler2".to_owned())
}

async fn handler3() -> Result<String, StatusCode> {
    Ok("handler3".to_owned())
}

async fn handle_too_many_requests(err: BoxError) -> (StatusCode, String) {
    (
        StatusCode::TOO_MANY_REQUESTS,
        format!("To many requests: {}", err)
    )
}

What am I doing wrong? Or is that information outdated for new versions of axum and tower?

axum version

0.7.4

[davidpdrsn]

Whats the problem?

[jplatte]

It seems likely that the problem is that the RateLimitLayer is cloned frequently by axum, maybe once per connection or even per request 🤔
That layer can be cloned, but creating multiple RateLimit services from clones of the same RateLimitLayer don't end up sharing the counts.

I wonder what the behavior even should be for axum there. We currently require layers to be Clone and I think it makes sense for performance, so multiple connections don't share layer instances. But then any service where cloning one instance of it a bunch of times vs. sharing one makes a semantic difference run into issues like this.

[davidpdrsn]

Ah I see. I suppose the solution would be something like https://docs.rs/tower/0.4.13/tower/limit/concurrency/struct.GlobalConcurrencyLimitLayer.html where the layer uses some shared rate limit.

[dayvejones]

Hello!

The GlobalConcurrencyLimitLayer works, but does not fully replace the desired RateLimitLayer.
RateLimitLayer allowed a certain number of requests to be completed within a set time window, and rejected requests beyond the limit. And the GlobalConcurrencyLimitLayer sets the number of simultaneously executed requests. New requests wait for the previous ones to be completed.

So Towers RateLimitLayer cannot be used with the new versions of axum and now I need to create something like a global Arc<RwLock<HashMap<NaiveDateTime, u32>>> and work through the .with_state(state)?

[davidpdrsn]

The GlobalConcurrencyLimitLayer works, but does not fully replace the desired RateLimitLayer.

I didn’t mean that it’s a replacement. Just that how it shares state is the same technique one could apply to the rate limit.

So Towers RateLimitLayer cannot be used with the new versions of axum

Correct but axum has worked like this since 0.6 which came out over a year ago. So not recent.

[dayvejones]

Yeah, I understand it. I just checked the GlobalConcurrencyLimitLayer)

Rate liming is useful feature. It would be great to have an elegant solution if the tower is no longer helpful for this task 🙂