[General] Unstable Connection for Downloading Mainnet Snapshot

[Johnaverse]

Team or Project

No response

Environment

Mainnet

L2 block number

No response

Provide a brief description of the functionality you're trying to implement and the issue you are running into.

Description

Attempting to download the mainnet snapshot from https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz has resulted in repeated failures due to connection instability. Multiple download tools, including wget, aria2, and curl, have been tested, but all attempts have failed with different error messages.

Steps to Reproduce

Use any of the following commands to download the snapshot:

• wget https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz

• aria2c -x 8 -s 8 https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz

• curl -o output.sql.gz https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz

Observe the errors during or after the download attempt.

Observed Behavior

wget

2025-01-18 01:08:30 (82.9 MB/s) - Connection closed at byte 1649356767232. Giving up.

aria2

Download aborted. URI=https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz
Exception: [AbstractCommand.cc:351] errorCode=8 URI=https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz
-> [HttpResponse.cc:81] errorCode=8 Invalid range header. Request: 4870520832-2073867664761/2073867664762, Response: 0-2073867664761/2073867664762

curl

$ curl -o output.sql.gz  https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
30 1931G   30  583G    0     0  66.2M      0  8:17:41  2:30:15  5:47:26     0
curl: (56) OpenSSL SSL_read: Connection reset by peer, errno 104

Environment

Tools tested: wget, aria2, curl

Network connection: Stable with high bandwidth (10 Gbps)

Operating Systems: Tested on Ubuntu 20.04 and Debian 12

Suggested Resolution

• Investigate any firewall / security policy drop connection

• Consider providing a segmented download option or a torrent alternative for large files to improve reliability.

• Uploaded to a cloud provider and distributed to different geographical locations

Repo Link (Optional)

https://en-backups.matterlabs.dev/external_node_backup_2024_12_13T05_05.sql.gz

Additional Details

The file size is very large (~1.9 TB), which may be contributing to the instability.

The server appears to reset connections or fail to handle range requests properly during large file downloads.

Retrying the download results in the same failures, even when resuming partial downloads.

[sarahschwartz]

Hi @Johnaverse , thanks for sharing this! We will investigate and get back to you soon.

[sarahschwartz]

Just to follow up here, the team has identified the issue and are working on a solution. The issue should be fixed by the end of next week.

[katembfoust]

Hi @sarahschwartz ! Can you confirm when this is fixed so that we can try again when it is ready? Thank you!!