Problems with reading xarray-written netcdf file with C++ software

[stella-bourdin]

Hi,

I am using a C++ Software called TempestExtremes (https://github.com/ClimateGlobalChange/tempestextremes) for my research, which reads NetCDF files to identify tropical cyclones tracks based on a user-defined set of criteria.
For one of my criteria, I needed to compute a field from my original data, that I then wrote back into a NetCDF for it to be read by TempestExtremes.
I computed the field in python using xarray for manipulation. The file is good and the field looks good, but for some reason, TempestExtremes takes a lot more time to read it than any other file. I was wondering whether this was a known issue due to the encoding of the file or anything... ? The size of the file being similar to the others.

If it's useful, the new field and the file are computed using the code here : https://github.com/stella-bourdin/dynamicoPy/blob/cyclone_tracking/dynamicopy/compute.py and especially the compute_OWZ_from_files function.

Thank you for your help !
Stella

[keewis]

I'm not really familiar with the encoding part of xarray, but to help you we would need to compare the output of Dataset.info(). Could you post the output of:

xr.open_dataset("faster.nc").info()
xr.open_dataset("slower.nc").info()

where "slower.nc" is the file you wrote using xarray

[stella-bourdin]

Here it is for the fast one :

>>> xr.open_dataset(u).info()
xarray.Dataset {
dimensions:
        latitude = 721 ;
        level = 2 ;
        longitude = 1440 ;
        time = 124 ;

variables:
        float32 longitude(longitude) ;
                longitude:standard_name = longitude ;
                longitude:long_name = longitude ;
                longitude:units = degrees_east ;
                longitude:axis = X ;
        float32 latitude(latitude) ;
                latitude:standard_name = latitude ;
                latitude:long_name = latitude ;
                latitude:units = degrees_north ;
                latitude:axis = Y ;
        float64 level(level) ;
                level:standard_name = air_pressure ;
                level:long_name = pressure level ;
                level:units = millibars ;
                level:positive = down ;
                level:axis = Z ;
        datetime64[ns] time(time) ;
                time:standard_name = time ;
                time:long_name = time ;
        float32 u(time, level, latitude, longitude) ;
                u:standard_name = eastward_wind ;
                u:long_name = U component of wind ;
                u:units = m s**-1 ;
                u:level_desc = pressure level ;

// global attributes:
        :CDI = Climate Data Interface version 1.6.8 (http://mpimet.mpg.de/cdi) ;
        :history = Thu Mar 18 14:29:52 2021: cdo -select,level=500,850 /bdd/ERA5/NETCDF/GLOBAL_025/4xdaily/AN_PL/1996/u.199601.aphe5.GLOBAL_025.nc /data/sbourdin/OWZ_ERA5/tmp/u_199601.nc
Tue Apr 30 16:41:02 2019: ncatted -a history,global,d,, /scratch/ms/fr/asn/ERA5/AN_PL/1996/u.199601.aphe5.GLOBAL_025.nc ;
        :source = credit: ECMWF, COPERNICUS Climate Change Service data, available from the ESPRI/IPSL data center ;
        :Conventions = CF-1.6 ;
        :title = ERA5 reanalysis ;

And the slow one :

>>> xr.open_dataset(owz).info()
xarray.Dataset {
dimensions:
        latitude = 721 ;
        level = 2 ;
        longitude = 1440 ;
        time = 124 ;

variables:
        float32 longitude(longitude) ;
                longitude:standard_name = longitude ;
                longitude:long_name = longitude ;
                longitude:units = degrees_east ;
                longitude:axis = X ;
        float32 latitude(latitude) ;
                latitude:standard_name = latitude ;
                latitude:long_name = latitude ;
                latitude:units = degrees_north ;
                latitude:axis = Y ;
        float64 level(level) ;
                level:standard_name = air_pressure ;
                level:long_name = pressure level ;
                level:units = millibars ;
                level:positive = down ;
                level:axis = Z ;
        datetime64[ns] time(time) ;
                time:standard_name = time ;
                time:long_name = time ;
        float64 owz(time, level, latitude, longitude) ;

// global attributes:
        :units = s-1 ;
        :history = Thu Mar 18 14:53:13 2021: ncrename -v vo,owz owz_199601.nc owz_199601.nc ;

I now see that one is float 32 and the other float64, maybe it makes a difference, I will try to convert it.