"inverse" of xr.open_dataset(xr.backends.NetCDF4DataStore)?

[chiaral]

I am trying to run a code that does not use xarray (seriously, why tho?!?) and loads data with netCDF4 and handles the netCDF4._netCDF4.Dataset directly.

I need to transpose my netCDF data, though, the code wants time to be the last dimension. I could do this before running the code, but my first idea was to add a few lines of code: load the data in xarray, transpose, and re-output them as a netCDF4.Dataset.

Thanks to Ryan May and this SO answer I got the first part done:

    xrdataset = xr.open_dataset(xr.backends.NetCDF4DataStore(nc))
    xrdataset = xrdataset.transpose(['latitude','longitude','time'])

I now need the "inverse" of xr.open_dataset(xr.backends.NetCDF4DataStore(nc)) , if it exists! , something like

nc = something(xrdataset)

Is that possible? without any writing/reading from file?

Thanks!

PS: I did search the discussion forum before and didn't find any suitable answer, if I missed one apologies!

[TomNicholas]

There is a .dump_to_store method on Dataset which might do what you want? It's not listed in the public API though so use with caution...

EDIT: In fact why is it not explicitly listed in api.rst? Both load_store and dump_to_store are mentioned in the text on that page but not actually given their own hyperlinked entries. 😕

[chiaral]

I will try it out! this would be wonderful!

[chiaral]

I tried to use .dump_to_store - although w/o description I am not sure I am doing it right.
But I couldn't over write the pervious nc file, which I guess makes sense?
I had to write to disc and re-read it. Is there a way to avoid that?

Here is what I am doing:

import xarray as xr
import netCDF4
import numpy as np

# generate a dummy file and save it to then read it through netCDF4
ds = xr.Dataset(
     data_vars=dict(
         temperature=([ "time","lat", "lon"], np.random.rand(100,10,10)),
     ),
     coords=dict(
         lon=(["lon"], np.arange(0,10)),
         lat=(["lat"],  np.arange(0,10)),
         time= np.arange(0,100),
     ),
 )
ds.to_netcdf('dummy_file.nc')

# open the file with netCDF4
nc = netCDF4.Dataset('dummy_file.nc')

# pass it to xarray to transpose
ds = xr.open_dataset(xr.backends.NetCDF4DataStore(nc))
ds = ds.transpose('lat','lon','time')

# create a new file
ncfile = netCDF4.Dataset('new.nc',mode='w',format='NETCDF4_CLASSIC') 

# dump to store
ds.dump_to_store(xr.backends.NetCDF4DataStore(ncfile))

This above works perfectly. But again, I have to write to file.

If I try to overwrite nc I get the error that the shape is different from what expected. (I edited my previous statement).

This error makes sense, it might be necessary to write to file.

If I figure out something I will post here.

[kmuehlbauer]

You might try to create an in-memory netcdf-file, open that handle with NetCDF4DataStore and then dump to that.