ChunkedUploady to AWS S3 (Multipart Upload)

[peacockbah]

We have the desire to leverage chunked uploady to send multi-part uploads to S3 using signedURLs.

We assume it'll be fine to hit our backend to generate a presigned URL in the item start listener & keep track of...

How can we update the URL per chunk to work with these presigned URLs? (https://aws.amazon.com/blogs/compute/uploading-large-objects-to-amazon-s3-using-multipart-upload-and-transfer-acceleration/) AWS uses a presigned URL PER part on the update.

We're currently looking at something like this:

useChunkStartListener(({ item, chunk }) => {
    const partNumber = chunk.index + 1; // S3 part numbers start at 1
    const urls = presignedUrls[item.id];
    
    if (!urls || !urls[partNumber]) {
      console.error(`No pre-signed URL found for part ${partNumber}`);
      return;
    }
    
    // This is the key part - we're returning a custom URL and options
    // for this specific chunk
    return {
      url: urls[partNumber],
      sendOptions: {
        method: 'PUT', // S3 pre-signed URLs expect PUT
        headers: {
          'Content-Type': item.file.type,
        }
      }
    };
  });

Above is untested, trying to ask some questions before we go down this route :)
Thank you for your time!

[yoavniran]

hey @peacockbah
Exactly. The useChunkStartListener event hook is the way to go.

[peacockbah]

When you generate persigned URLs for a miltupart upload, you must tell AWS the amount of parts (e.g., the amount of chunks).
How would this be known before chunking occurs?

Taking the total size / chunk size you set on the GUI and hardcoding the server for the same?

[peacockbah]

Ahh, might've hit the real blocker here -

https://react-uploady.org/docs/packages/rpldy-chunked-uploady/#details

We have a use case where we may use the simple url or multiplepart, the sever will generate the correct url based on the file size.
In the event of a simple URL, we cannot send the Content-Range header, which chunked uploady does by default.

Is there any way to strip this out? This would allow us to use chunked-uploady to send a single chunk, or multiple chunks dynamically.

[peacockbah]

I ended up finding a solution forward- on the instance we use AWS S3 with presigned urls, I abandoned the chunkedUploady in favor of raw uploady and provide a custom sender.

[yoavniran]

@peacockbah
I will look at adding a param to control the range header. would that allow you to use the chunked-sender (chunked-uploady) again?

[peacockbah]

For simple presigned URLs, you cannot have any unknown headers, nor can you sign them on the server. There are very specific headers you must provide - I think having control over the headers entirely would be necessary.