Skip to content

Encounter InternalServerError and InvalidPartOrder when uploading to S3 storage #1315

New issue
New issue
Open
Open
Encounter InternalServerError and InvalidPartOrder when uploading to S3 storage#1315
Labels
bug
@thinhpham

Description

@thinhpham
thinhpham
opened on Sep 3, 2025

Hello,

Describe the bug
I use S3 as the storage for tusd. When uploading to S3, anything smaller than 180MB works fine. However, larger files caused tusd to return InternalServerError which was caused by S3 returning InvalidPartOrder error.

To Reproduce
Steps to reproduce the behavior:
tusd is running in a Docker container that was execute with the following command:

CMD ["-hooks-dir", "/srv/tusd-hooks", "-upload-dir", "/srv/uploads", "-max-size", "52428800", "-s3-bucket", "tusd", "-s3-object-prefix", "uploads", "-s3-part-size", "52428800", "-s3-min-part-size", "52428800", "-disable-download", "-behind-proxy", "-verbose"]

Expected behavior
Should upload without any S3 error

Setup details
Please provide following details, if applicable to your situation:

  • Operating System: macOS running tusd in a Docker container
  • Used tusd version: tusproject/tusd:latest
  • Used tusd data storage: AWS S3
  • Used tusd configuration: -hooks-dir "/srv/tusd-hooks" -upload-dir "/srv/uploads" -max-size 52428800 -s3-bucket "tusd" -s3-object-prefix "uploads" -s3-part-size 52428800 -s3-min-part-size 52428800 -disable-download -behind-proxy -verbose
  • Used tus client library: Uppy.js

Logs

2025/09/03 21:40:44.667216 Using 's3://tusd' as S3 bucket for storage.
2025/09/03 21:40:44.667473 Using 476.84MB as maximum size.
2025/09/03 21:40:44.667588 Using '/srv/tusd-hooks' for hooks
2025/09/03 21:40:44.667618 Enabled hook events: pre-create, post-create, post-receive, post-terminate, post-finish
2025/09/03 21:40:44.667628 Supported tus extensions: creation,creation-with-upload,termination,concatenation,creation-defer-length
2025/09/03 21:40:44.667635 Using 0.0.0.0:8080 as address to listen.
2025/09/03 21:40:44.667643 Using /files/ as the base path.
2025/09/03 21:40:44.667674 Using /metrics as the metrics path.
2025/09/03 21:40:44.667905 You can now upload files to: http://[::]:8080/files/
2025/09/03 21:40:59.721493 level=INFO event=RequestIncoming method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:40:59.981639 level=INFO event=ResponseOutgoing method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=200 body=""
2025/09/03 21:40:59.985999 level=INFO event=RequestIncoming method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:41:00.063905 level=INFO event=ChunkWriteStart method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm maxSize=5242880 offset=268922212
2025/09/03 21:41:01.065962 level=DEBUG event=HookInvocationStart type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
Upload c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm is at 100% (274165092/274165092)
2025/09/03 21:41:01.067425 level=DEBUG event=HookInvocationFinish type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
2025/09/03 21:41:02.385453 level=INFO event=ChunkWriteComplete method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm bytesWritten=5242880
2025/09/03 21:41:02.470093 level=ERROR event=InternalServerError method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm message="operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: CG61ZKMD9B7WYJEY, HostID: meGyMvasGfY0jBHf4j90p4XU9MisnlViB+t2CSDcrd/lVX+2ZGL1UtjN/PdWTsoC7C6NZEfsFHU=, api error InvalidPartOrder: The list of parts was not in ascending order. Parts must be ordered by part number."
2025/09/03 21:41:02.470181 level=INFO event=ResponseOutgoing method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=500 body="ERR_INTERNAL_SERVER_ERROR: operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: CG61ZKMD9B7WYJEY, HostID: meGyMvasGfY0jBHf4j90p4XU9MisnlViB+t2CSDcrd/lVX+2ZGL1UtjN/PdWTsoC7C6NZEfsFHU=, api error InvalidPartOrder: The list of parts was not in ascending order. Parts must be ordered by part number.\n"
2025/09/03 21:41:02.574644 level=INFO event=RequestIncoming method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:41:02.724995 level=INFO event=ResponseOutgoing method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=200 body=""
2025/09/03 21:41:02.728124 level=INFO event=RequestIncoming method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:41:02.805040 level=INFO event=ChunkWriteStart method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm maxSize=1535332 offset=272629760
2025/09/03 21:41:03.654210 level=INFO event=ChunkWriteComplete method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm bytesWritten=1535332
2025/09/03 21:41:03.737479 level=ERROR event=InternalServerError method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm message="operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: MMP6K86GZYHH67Y6, HostID: DLYNAlhjZtzY7sOQAGbddjJpgikRICPJGFODUI50vaSwGAoNFsR1BiM8nhxG3RTvMzhItCYoEok=, api error InvalidPartOrder: The list of parts was not in ascending order. Parts must be ordered by part number."
2025/09/03 21:41:03.737552 level=INFO event=ResponseOutgoing method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=500 body="ERR_INTERNAL_SERVER_ERROR: operation error S3: CompleteMultipartUpload, https response error StatusCode: 400, RequestID: MMP6K86GZYHH67Y6, HostID: DLYNAlhjZtzY7sOQAGbddjJpgikRICPJGFODUI50vaSwGAoNFsR1BiM8nhxG3RTvMzhItCYoEok=, api error InvalidPartOrder: The list of parts was not in ascending order. Parts must be ordered by part number.\n"
2025/09/03 21:41:03.806884 level=DEBUG event=HookInvocationStart type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
Upload c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm is at 100% (274165092/274165092)
2025/09/03 21:41:03.807742 level=DEBUG event=HookInvocationFinish type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
2025/09/03 21:41:03.840263 level=INFO event=RequestIncoming method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:41:03.960188 level=INFO event=ResponseOutgoing method=HEAD path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=200 body=""
2025/09/03 21:41:03.963284 level=INFO event=RequestIncoming method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId=""
2025/09/03 21:41:04.032562 level=INFO event=ChunkWriteStart method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm maxSize=5242880 offset=268922212
2025/09/03 21:41:05.033122 level=DEBUG event=HookInvocationStart type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
Upload c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm is at 100% (274165092/274165092)
2025/09/03 21:41:05.034467 level=DEBUG event=HookInvocationFinish type=post-receive id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm
^C2025/09/03 21:41:06.036337 Received interrupt signal. Shutting down tusd...
2025/09/03 21:41:06.403656 level=ERROR event=BodyReadError method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm error="ERR_SERVER_SHUTDOWN: request has been interrupted because the server is shutting down"
2025/09/03 21:41:06.403737 level=INFO event=ChunkWriteComplete method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm bytesWritten=5242880
2025/09/03 21:41:06.497958 level=INFO event=ResponseOutgoing method=PATCH path=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm requestId="" id=c27364b1d4e9f7d0c6090e376f2bf42b+C7rNO_3A9rl_FOtXBD0BQxUqIBsrVSLtOXmbwwe2zNAtQT4FK0cCgDNpKlNOWBw.b0cNV_ssmCqRK3AjbzSGWIqdv2dtZZIpl77mWNFI1_KHgNlWwk_s_ADWMUwyUPhm status=503 body="ERR_SERVER_SHUTDOWN: request has been interrupted because the server is shutting down\n

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions