Multiple variables sharing part of the grid result in colossal NetCDF when merged

[marcosrdac]

Hello everyone,

I have an original 3D NetCDF dataset of 16GB.
In another file, I have a 3D portion of the same NetCDF (~3GB), along with some other variables (i.e.: 3D segmentation, other derived variables).

When I merge them into an only file, I get a dataset of 46GB, (not ~19GB, which is what I expected).
I believe the NaN padding that appear after the merge operation is taking a lot more memory than needed.

Is there any way to avoid such kind of disk inefficiency when saving xarray files as NetCDF?