serve a new file from an endpoint [getting 0 bytes file]

[olekspickle]

Summary

Hey! I am trying to form a zip file(created with zip crate) with encrypted file on request to endpoint and serve it.
I tried all approaches in #608 and can't figure out what is wrong.

best case scenario - I get zip file that says 'not a zip archive' when opened.

I have something like this

setup

use std::{fs::File, net::TcpListener};
use tokio::net::TcpListener;
use zip::{write::SimpleFileOptions, AesMode, CompressionMethod};
use axum::body::Body;
use hyper::{header, HeaderMap};

async fn create_zip() -> Result<Vec<u8>> {
let file = File::create("lol.zip")?;
    let mut zip = zip::ZipWriter::new(file);
    let id = env::var("SECRET").unwrap_or_else(|_| "real-big-secret".into());
    let opts = SimpleFileOptions::default()
        .compression_method(CompressionMethod::Bzip2)
        .with_aes_encryption(AesMode::Aes256, &id);

    zip.start_file(FILE, opts)?;
    zip.write_all(b"==============*SAVE-ME-FERRIS*==============")?;
    
    // reading file right after writing is for sure stupid, but I don't really care about that at the moment
    zip.finish()?;
    let file = fs::read("lol.zip")?;
    Ok(file)
}

pub async fn fetch_zip() -> impl IntoResponse {
    let zip = create_zip().await.unwrap_or_default();
    let body = Body::from(zip);

    let mut headers = HeaderMap::new();
    headers.insert(header::CONTENT_TYPE, "application/zip".parse().unwrap());
    headers.insert(
        header::CONTENT_DISPOSITION,
        format!("attachment; filename=\"lol.zip\"").parse().unwrap(),
    );

    (headers, body).into_response()
}

fn main() {
    let router = Router::new().route("/fetch-zip", get(fetch_zip));
    let listener = TcpListener::bind("0.0.0.0:8000").await?;
    axum::serve(listener, router).await?;
}

this gives me an empty lol.zip which is eager to tell me that it is not a zip file

axum version

0.7.5

[mladedav]

What you're describing sounds more like a problem with the creation of zip file rather than an axum issue. Can you verify that the lol.zip file has any contents and that your zip variable in the handler is non-empty?

[olekspickle]

yep, I tried to run zip creation separately and it worked perfectly fine, opened zip, entered pass, see the text all good.
but served with axum it produces this not a zip file thingy

[mladedav]

And can you verify that the Vec is non-empty? I don't know how the library works if there can be some races concerning flushing, but I really doubt this is an issue with axum.

[olekspickle]

ok, seems there is indeed some problem - after dockerization and packaging I see 0 bytes in handler. It's weird though because when I call the same function in test - I see the zip is 12kB and I can access the data in it no problem

[olekspickle]

Ok, so I think what is happening is that the enpoint returning before all writing operation actually finish.
Not sure why though: I do await the function in handler, no idea why would axum just skip that work

Is there any example with the heavy deferred loading?..

[olekspickle]

Ok, I tested some more. Running locally with cargo run [--release] works fine - I get the zip, can decrypt the file in it and all, but when I pack the app into container - I am looking at the same endpoint returning 0 byte zip file.

Is this a common pitfall with docker?

[mladedav]

No idea, this sounds like an issue with the zip library which I have never used.

But what's probably causing you more problems that it's worth is this line: let zip = create_zip().await.unwrap_or_default();. The zip creation fails for some reason but you're just discarding the error and using an empty vector. Once you handle the errors you'll know what's wrong.

[olekspickle]

Ok I got what was happening: I was calling docker and journald commands to gather debug info and write it to zip file and those commands were not available in container itself. It failed with a stupid

Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }))

which confused the hell out of me.