Include retry logic and logging for scenario: "0 B/s was observed" error #176
Description
While running an s3invsync backup, the follow error was observed:
Error: failed downloading contents for s3://linc-brain-mit-prod-us-east-2/blobs/290/e07/290e070d-9681-41d4-a0f0-966df911e943
Caused by:
0: streaming error
1: minimum throughput was specified at 1 B/s, but throughput of 0 B/s was observed
While it is hard to trace whether underlying infrastructure may have caused zero bytes to be transferred at that time, s3invsync may be a much more robust tool if retry logic was in place for this error state
The logs show this error occurring 3 times, prior to the script shutting down.
For confirmation, the given asset does exist in the S3 bucket, and is of size > 0B:
aaronkanzer@Aarons-MacBook-Pro ~ % aws s3 ls s3://linc-brain-mit-prod-us-east-2/blobs/290/e07/290e070d-9681-41d4-a0f0-966df911e943 --human-readable --summarize
2024-03-13 23:46:46 7.5 MiB 290e070d-9681-41d4-a0f0-966df911e943
Total Objects: 1
Total Size: 7.5 MiB
Note: Upon re-running the script, this asset was properly backed up.