File uploads streaming method, chunk issue

[steteb]

Hello,

I am trying the example:

https://mongoose.ws/tutorials/file-uploads/#streaming-method

I am using Mongoose v. 7.6

first of all, I noticed that it doesn't seem to be true that the last chunk is the one with len 0, so if I try the example as it is, the mg_http_reply(c, 200, "", "ok (chunked)\n") is never sent.

But the big problem I encountered is that the first time I upload a file I receive all the chunks correctly. If I try a second time, the first chunk seems to be not correct and it contains extra data (data of the POST request or http header, I don't know exactly) respect the file data.

Here below the image of the content of mg_http_message hm structure in the OK case:

and in the NOK case:

Any explanation about this behaviour ?

Thank you

[cpq]

@steteb Can you elaborate on "If I try second time", please? What second time means? Can you share an example, please?

[steteb]

For the second time I mean that after uploaded a file and received "Uploaded x bytes" with the streaming method example, I press again the button and select another file to upload. This second time I receive in the first chunk not the beginning of the file, but extra data as indicated in the image above.

I call mg_http_delete_chunk() for every chunk and on the last I send mg_http_reply(c, 200, "", "ok (chunked)\n"); and in fact I received on the browser correctly the message "Uploaded x bytes".

Moreover, as I said, the condition to identify the last chunk doesn't seem to be if (hm->chunk.len == 0), it is never true. To identify the last chunk I sum the chunk.len and when the total bytes received are == body.len I know the entire file has been received and I can send the reply.

[cpq]

@steteb thanks. Does the same reproduce by the latest HEAD on your system? I can't seem to be able to reproduce that (Linux).

[pmeskoni]

I believe this is the same issue I was having, described in the first part of #1499
My experience was that the first message received had chunk.len = 0 and when I went through each message deleting the chunk each time. I still could not determine the end of the first file and start of the second without counting up the expected files size myself.

I ended up abandoning that approach. Instead I resized the read buffer to be large enough for the entire file when I got the first chunk notification by calling mg_iobuf_resize() with the expected file size, then let Mongoose proceed normally.

[LorenzoCarretta]

Hello,

I'm actually incurring in the same issue as the poster steteb said.
What I'm experiencing is that after the last chunk, after I used "mg_http_delete_chunk", mongoose doesn't recognize that the last chunk (0\r\n) has arrived, so my application layer doesn't know that the file transfer is complete and that connection should be closed.

I managed to handle the "transfer complete" counting the number of bytes received against Content-Length header, so that is fine, but the issue is that I cannot gracefully close connection because the receiving buffer would still contain last POST.
Also whatever other call I would do from the same browser window would get on the same connection used for this POST and it would break my application since it would see the new request but the last one in the c->recv buffer (you can see it in the memory screenshot below).

I would like to know if this bug's been handled and/or if there is some solution to handling the end of transfer.

[scaprile]

Your Mongoose version ?
See #1656 and https://mongoose.ws/tutorials/file-uploads/#binary-upload

[LorenzoCarretta]

I'm using version 7.6.
What I see is that the recv buffer keeps the headers (I imagine that is so that application can know the URL, query params, ecc.) and adding the chunk at the end, then the mg_http_delete_chunk only removes the chunk part and keeps the headers, but after the end of the file how can I delete the headers too from receive buffer?

[scaprile]

If you see anything wrong with latest please open an issue.

[scaprile]

This has already been fixed for 7.8 (see #1656 and https://mongoose.ws/tutorials/file-uploads/#binary-upload)

[LorenzoCarretta]

Thank you we'll upgrade to the new version 7.8.