Tower Http RequestDecompressionLayer Example request #2031

theelderbeever · 2023-06-02T22:52:43Z

theelderbeever
Jun 2, 2023

Hopefully I am just missing something obvious however, I am trying to build a thin api that receives metrics data. To this end the api needs to be able to handle compressed request bodies that are hinted by the Content-Encoding: deflate (or others) header. I have been looking at the RequestDecompressionLayer and trying to implement it. However, I keep getting a trait impl error that I can't make sense of and have no clue how to fix. Below is some example code to make the error show up. Its taken almost exactly from the tower_http example. If you remove the layer it is just a post echo server right now. However, I would like to be able to send a compressed payload and then read it as json. Any help would be greatly appreciated.

use axum::{
    body::Body,
    http::{Request, Response},
    routing::post,
    BoxError, Json, Router,
};
use bytes::BytesMut;
use http_body::Body as _;
use serde_json::Value;
use std::str::FromStr;
use tower::{service_fn, ServiceBuilder};
use tower_http::decompression::{DecompressionBody, RequestDecompressionLayer};

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let app: Router = Router::new().route("/", post(root)).layer(
        ServiceBuilder::new()
            .layer(RequestDecompressionLayer::new())
            .service(service_fn(decompression_handler)),
    );

    let addr = std::net::SocketAddr::from_str(&format!("{}:{}", "127.0.0.1", 8000)).unwrap();
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn decompression_handler(
    mut req: Request<DecompressionBody<Body>>,
) -> Result<Response<Body>, BoxError> {
    let mut data = BytesMut::new();

    while let Some(chunk) = req.body_mut().data().await {
        let chunk = chunk?;
        data.extend_from_slice(&chunk[..]);
    }
    assert_eq!(data.freeze().to_vec(), b"Hello?");
    Ok(Response::new(Body::from("Hello, World!")))
}

async fn root(Json(value): Json<Value>) -> Json<Value> {
    Json(value)
}

The Error

the trait bound `tower_http::decompression::RequestDecompression<tower::util::ServiceFn<fn(axum::http::Request<tower_http::decompression::DecompressionBody<axum::body::Body>>) -> impl std::future::Future<Output = std::result::Result<axum::http::Response<axum::body::Body>, std::boxed::Box<(dyn std::error::Error + std::marker::Send + std::marker::Sync + 'static)>>> {decompression_handler}>>: tower::Layer<axum::routing::Route<_>>` is not satisfied
the following other types implement trait `tower::Layer<S>`:
  <&'a T as tower::Layer<S>>
  <() as tower::Layer<S>>
  <(L1, L2) as tower::Layer<S>>
  <(L1, L2, L3) as tower::Layer<S>>
  <(L1, L2, L3, L4) as tower::Layer<S>>
  <(L1, L2, L3, L4, L5) as tower::Layer<S>>
  <(L1, L2, L3, L4, L5, L6) as tower::Layer<S>>
  <(L1, L2, L3, L4, L5, L6, L7) as tower::Layer<S>>
and 41 othersrustc[Click for full compiler diagnostic](rust-analyzer-diagnostics-view:/diagnostic%20message%20%5B0%5D?0#file%3A%2F%2F%2FUsers%2Ftaylorbeever%2Fgit%2Ftraining%2Frust%2Faxum-api%2Fsrc%2Fmain.rs)
main.rs(16, 60): required by a bound introduced by this call
mod.rs(228, 12): required by a bound in `axum::Router::<S, B>::layer`

davidpdrsn · 2023-06-05T10:14:44Z

davidpdrsn
Jun 5, 2023
Collaborator

I don't really understand what you're trying to do. When you call .service(...) on a ServiceBuilder you get a service, not a middleware. Thus you cannot pass it do Router::layer as that takes middleware.

If you just wanna decompress all requests you need to use HandleErrorLayer to handle errors since RequestDecompressionLayer can fail:

use axum::{
    body::Body,
    error_handling::HandleErrorLayer,
    http::{Request, Response, StatusCode},
    routing::post,
    BoxError, Json, Router,
};
use bytes::BytesMut;
use http_body::Body as _;
use serde_json::Value;
use std::str::FromStr;
use tower::{service_fn, ServiceBuilder};
use tower_http::decompression::{DecompressionBody, RequestDecompressionLayer};

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let app: Router = Router::new().route("/", post(root)).layer(
        ServiceBuilder::new()
            .layer(HandleErrorLayer::new(|error: BoxError| async move {
                (StatusCode::INTERNAL_SERVER_ERROR, "Unhandled server error")
            }))
            .layer(RequestDecompressionLayer::new()),
    );

    let addr = std::net::SocketAddr::from_str(&format!("{}:{}", "127.0.0.1", 8000)).unwrap();
    axum::Server::bind(&addr)
        .serve(app.into_make_service())
        .await
        .unwrap();
}

async fn root(Json(value): Json<Value>) -> Json<Value> {
    Json(value)
}

8 replies

theelderbeever Jun 6, 2023
Author

My build_router most definitely does return Router.... Any recommendations on a better pattern for constructing subrouters or is it really just down to the return type of the function? Or would it be better to use nest_service and return a Service object instead of a Router?

pub fn build_router(state: &ApplicationState) -> Router<ApplicationState> {
    let _debug = String::from("debug");
    if let Ok(_debug) = std::env::var("RUST_LOG") {
        std::env::set_var(
            "RUST_LOG",
            "example_print_request_response=debug,tower_http=debug",
        )
    }

    Router::new()
        .route("/api/v2/series", post(post_dd_agent_timeseries))
        .route_layer(middleware::from_fn(authenticate))
        .layer(
            ServiceBuilder::new()
                .layer(HandleErrorLayer::new(handle_error))
                .layer(RequestDecompressionLayer::new()),
        )
        .layer(DefaultBodyLimit::disable())
        .layer(Extension(state.clone()))
        .with_state(state.clone())
        .route("/intake/", post(blackhole))
        .route("/api/v1/metadata", post(blackhole))
        .route("/api/v1/check_run", post(blackhole))
}

ironspecs Aug 15, 2023

Any follow up on this?

I also return Router. Is there a better way?

davidpdrsn Aug 15, 2023
Collaborator

@TakenPilot please open a new discussion with your problem. Can't help you without any context.

ironspecs Aug 16, 2023

My build_router most definitely does return Router.... Any recommendations on a better pattern for constructing subrouters or is it really just down to the return type of the function? Or would it be better to use nest_service and return a Service object instead of a Router?

The context is just looking for follow up on their post, since I have the same question as them.

davidpdrsn Aug 16, 2023
Collaborator

Alright. Returning Router is the way to go but like I said you need to use the right type parameters.

tisonkun · 2024-03-13T13:56:22Z

tisonkun
Mar 13, 2024

I get:

   Compiling servers v0.7.0 (/Users/tison/GreptimeWorkspace/greptimedb/src/servers)
error[E0277]: the trait bound `Infallible: std::convert::From<Box<(dyn StdError + std::marker::Send + std::marker::Sync + 'static)>>` is not satisfied
   --> src/servers/src/http.rs:702:20
    |
702 |             .layer(RequestDecompressionLayer::new().gzip(true))
    |              ----- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::convert::From<Box<(dyn StdError + std::marker::Send + std::marker::Sync + 'static)>>` is not implemented for `Infallible`
    |              |
    |              required by a bound introduced by this call
    |
    = help: the trait `std::convert::From<!>` is implemented for `Infallible`
    = help: for that trait implementation, expected `!`, found `Box<(dyn StdError + std::marker::Send + std::marker::Sync + 'static)>`
    = note: required for `Box<(dyn StdError + std::marker::Send + std::marker::Sync + 'static)>` to implement `Into<Infallible>`
note: required by a bound in `AxumRouter::<S, B>::layer`
   --> /Users/tison/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/mod.rs:238:62
    |
233 |     pub fn layer<L, NewReqBody>(self, layer: L) -> Router<S, NewReqBody>
    |            ----- required by a bound in this associated function
...
238 |         <L::Service as Service<Request<NewReqBody>>>::Error: Into<Infallible> + 'static,
    |                                                              ^^^^^^^^^^^^^^^^ required by this bound in `Router::<S, B>::layer`

How do you get RequestDecompressionLayer successfully configured ...

1 reply

tisonkun May 24, 2024

A solution - #2647 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Tower Http RequestDecompressionLayer Example request #2031

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 9 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Tower Http RequestDecompressionLayer Example request #2031

Uh oh!

theelderbeever Jun 2, 2023

Replies: 2 comments · 9 replies

Uh oh!

davidpdrsn Jun 5, 2023 Collaborator

Uh oh!

Uh oh!

theelderbeever Jun 6, 2023 Author

Uh oh!

ironspecs Aug 15, 2023

Uh oh!

davidpdrsn Aug 15, 2023 Collaborator

Uh oh!

ironspecs Aug 16, 2023

Uh oh!

davidpdrsn Aug 16, 2023 Collaborator

Uh oh!

tisonkun Mar 13, 2024

Uh oh!

tisonkun May 24, 2024

theelderbeever
Jun 2, 2023

Replies: 2 comments 9 replies

davidpdrsn
Jun 5, 2023
Collaborator

theelderbeever Jun 6, 2023
Author

davidpdrsn Aug 15, 2023
Collaborator

davidpdrsn Aug 16, 2023
Collaborator

tisonkun
Mar 13, 2024