Skyplane feature requests

[sarahwooders]

Help us plan what new features to include in Skyplane's future releases! It'd be extra helpful if you can describe your potential use-cases with Skyplane.

[TomNicholas]

Use case
A lot of scientists (particularly Climate Scientists) are now sharing their data via public cloud buckets (e.g. ERA-5 on GCP, see this paper for rationale). However, transferring data from large numerical simulations produced on HPC systems is often really challenging. I want to use skyplane to transfer large amounts of data from a HPC local filesystem to the cloud.

Features

• I want to be able to use Skyplane locally on a HPC filesystem (e.g. Lustre) to transfer TBs (or even PBs) of data to the cloud.
• Even better would be if I could call Skyplane from Python, to integrate with existing tooling for moving scientific datasets to the cloud (which currently can only pull data from other public data portals elsewhere on the internet.)

(from #924)