Microsoft Planatary Computer and the AWS NAT Gateway cost

[mehran66]

I have been fetching Sentinel and Naip imagery from Microsoft Planetary Computer and processing them using a local Dask cluster .... the AWS cost has surged noticeably.

Upon investigating the spike in costs is related to the AWS NAT Gateway usage. I learned that the NAT Gateway facilitates instances in a private subnet to initiate outbound IPv4 traffic to the internet, and in my scenario, it seems to be heavily utilized due to the data transfer demands of the satellite data fetching process.

I am reaching out to get any insights on this matter and to see if there is any potential optimizations or alternative architectures to mitigate such cost on AWS. Also I was wondering if switching to Azure VMs can help reducing the NAT Gateway cost given that the data is coming from Azure data storage?

[TomAugspurger]

Also I was wondering if switching to Azure VMs can help reducing the NAT Gateway cost given that the data is coming from Azure data storage?

I'd recommend this. I'm not 100% sure, but generally speaking within-region traffic is lower cost. In addition, you'll see better performance.

[mehran66]

Thanks @TomAugspurger

found this note as well:
https://planetarycomputer.microsoft.com/docs/quickstarts/storage/#:~:text=Note%20that%20your%20storage%20and%20compute%20should%20be%20located%20in%20the%20same%20Azure%20region.%20The%20Planetary%20Computer%20Hub%20and%20data%20reside%20in%20the%20West%20Europe%20region.