Why not just use Zip downloads?

[TogiFerretFerret]

They use relatively consistent naming - it should be possible to create automatic name-link mappings and download and unzip.

[TogiFerretFerret]

Every testcase zip along with info about the file (the full zip contents completely embedded) fits under 3 gb.

[bqi343]

zip downloads for what?

[TogiFerretFerret]

test data

[TogiFerretFerret]

btw, its pretty easy to write a script in python that uses requests_html and bs4 to scrape all test data and question info.

[TogiFerretFerret]

in total, with all the data and test data its less than 3gb