Getting NaN values depending on different subsets of same data

[tlsw231]

I am facing a weird error where depending on how I subset the same number of files, I end up getting NaN values in one case and not the other. Unfortunately, I don't know how to create a MVE for this issue. Here is what I am trying to do:

1. I have a set of netcdf files one per day for the year 2014 (365 total), say, files = {file0, file1,....file364}. The file has a bunch of variables with dimensions (time,latitude,longitude). The time dimension is a datetime64 object.

2. I create a couple of datasets splitting the 365 files into two sets: fileA={file0...,file338} and fileB={file339....file364}.

ds1=xr.open_mfdataset(fileA,parallel=True,data_vars='minimal',concat_dim='time',
                             chunks='auto',coords='minimal',compat='override')
ds2=xr.open_mfdataset(fileB,parallel=True,data_vars='minimal',concat_dim='time',
                             chunks='auto',coords='minimal',compat='override')

When I plot the timeseries of a variable at some (lat,lon), I get something like the picture below. The x-axis in the left plot starts from Jan. 1, as it should. The plot on the right shows continuity with the left plot, which also makes sense.

[The titles for the plots should read "first 339 files" and "files 340-365".]

3. I then try to create a dataset using the set fileC={file0,file1,....file339}. This is basically the set fileA above combined with the immediate next file in succession, which is file339.

ds3=xr.open_mfdataset(fileC,parallel=True,data_vars='minimal',concat_dim='time',
                                      chunks='auto',coords='minimal',compat='override')

Here is what the timeseries for the same variable at the same (lat,lon) looks like:

The x-axis now has shrunk remarkably and incorrectly starts from Dec. 6th, not Jan. 1st. This is because all values prior to Dec. 6th have been (incorrectly) read in as NaN! But the first plot shows values prior to Dec. 6th are not NaN.

I am baffled why my results should pivot on the inclusion of one extra file (file339) when file339 by itself is perfectly okay.

Potential issue?

I ran ncdump -h on files 338/339. The global attributes show the tiling structure changes right at file339. All prior files have a tiling of 020x018 but file339 and its successors have a tiling of 030x024. Is this messing up the open_mfdataset step when I try to combine all 365 files into one dataset?

[tlsw231]

Okay, I found out what is the problem, nothing to do with the tiling. It turns out the output from file339 onwards has an additional six scalar constants in each netcdf file. I am now doing the following:

ds1=xr.open_mfdataset(files[0:339],.....) # other keywords go inside parentheses
ds2=xr.open_mfdataset(files[339:],.......)
ds = xr.concat([ds1,ds2],'time')

That seems to be sufficient. I did not even have to drop those six constants from file339 onwards. Just splitting the datasets and concatenating later works. [Probably related to this issue?]