convert_calendar 360_day with offset start and end dates.

[ollie-bell]

I have an observational dataset on standard 365/366 day calendar, and a modelled dataset on the same spatial grid but with a 360 day calendar. Both datasets are daily frequency. I need to convert the observational dataset calendar to 360_day in order to do bias adjustment on the modelled dataset.

The modelled data has a start date of 1980-12-01 and end date of 2080-11-30, and when dealing with temporal subsets the processing I am doing makes sense to keep 12-01 and 11-30 start and end dates for both the observational and modelled datasets, e.g. 1980-12-01 to 1990-11-30:

>>> ds = xr.open_mfdataset(paths, use_cftime=True).sel(time=slice("1980-12-01", "1990-11-30"))
>>> ds.time
<xarray.DataArray 'time' (time: 3652)>
array([cftime.DatetimeGregorian(1980, 12, 1, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(1980, 12, 2, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(1980, 12, 3, 12, 0, 0, 0, has_year_zero=False),
       ...,
       cftime.DatetimeGregorian(1990, 11, 28, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(1990, 11, 29, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(1990, 11, 30, 12, 0, 0, 0, has_year_zero=False)],
      dtype=object)
Coordinates:
  * time     (time) object 1980-12-01 12:00:00 ... 1990-11-30 12:00:00
Attributes:
    axis:           T
    bounds:         time_bnds
    standard_name:  time

However, when converting the observational dataset calendar to 360_day using convert_calendar with these start and end dates, it returns a time coordinate over 1980-11-30 to 1990-11-29 (instead of the desired 1980-12-01 to 1990-11-30).

>>> ds.convert_calendar("360_day", align_on="year").time
<xarray.DataArray 'time' (time: 3600)>
array([cftime.Datetime360Day(1980, 11, 30, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 2, 12, 0, 0, 0, has_year_zero=True),
       ...,
       cftime.Datetime360Day(1990, 11, 27, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 29, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 1980-11-30 12:00:00 ... 1990-11-29 12:00:00
Attributes:
    axis:           T
    bounds:         time_bnds
    standard_name:  time

If this is indeed expected behaviour and isn't a bug in convert_calendar, then what is an appropriate way to deal with this?

e.g. I could take a hacky approach and just increment the resulting time coordinate by 1 day?

>>> from datetime import timedelta
>>> ds = ds.convert_calendar("360_day", align_on="year")
>>> ds = ds.assign_coords(time=ds.time.values+timedelta(days=1))
>>> ds.time
<xarray.DataArray 'time' (time: 3600)>
array([cftime.Datetime360Day(1980, 12, 1, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 2, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 3, 12, 0, 0, 0, has_year_zero=True),
       ...,
       cftime.Datetime360Day(1990, 11, 28, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 29, 12, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 11, 30, 12, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 1980-12-01 12:00:00 ... 1980-11-30 12:00:00
Attributes:
    axis:           T
    bounds:         time_bnds
    standard_name:  time

[keewis]

my knowledge of calendar conversions is pretty minimal, but I'm guessing that this is the expected behavior of align_on="year". If you set align_on="date", you get the expected dates:

In [1]: import xarray as xr

In [2]: times = xr.date_range("1980-12-01", "1990-11-30")
   ...: ds = xr.Dataset(coords={"time": times})
   ...: ds.time
Out[2]: 
<xarray.DataArray 'time' (time: 3652)>
array(['1980-12-01T00:00:00.000000000', '1980-12-02T00:00:00.000000000',
       '1980-12-03T00:00:00.000000000', ..., '1990-11-28T00:00:00.000000000',
       '1990-11-29T00:00:00.000000000', '1990-11-30T00:00:00.000000000'],
      dtype='datetime64[ns]')
Coordinates:
  * time     (time) datetime64[ns] 1980-12-01 1980-12-02 ... 1990-11-30

In [3]: ds.convert_calendar("360_day", align_on="year").time
Out[3]: 
<xarray.DataArray 'time' (time: 3600)>
array([cftime.Datetime360Day(1980, 11, 30, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 2, 0, 0, 0, 0, has_year_zero=True), ...,
       cftime.Datetime360Day(1990, 11, 27, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 28, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 29, 0, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 1980-11-30 00:00:00 ... 1990-11-29 00:00:00

In [4]: ds.convert_calendar("360_day", align_on="date").time
Out[4]: 
<xarray.DataArray 'time' (time: 3582)>
array([cftime.Datetime360Day(1980, 12, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 2, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1980, 12, 3, 0, 0, 0, 0, has_year_zero=True), ...,
       cftime.Datetime360Day(1990, 11, 28, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 29, 0, 0, 0, 0, has_year_zero=True),
       cftime.Datetime360Day(1990, 11, 30, 0, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 1980-12-01 00:00:00 ... 1990-11-30 00:00:00

cc @spencerkclark, @aulemahal

[aulemahal]

AFAIU the question, this is the correct answer.
align_on='year' will keep your times at the same "year-fraction", but that transforms the month-day information. align_on='date' keeps the month-day information.

The caveat is that with 'date', you lose invalid dates (Feb 30th and 29th). With 'year', you don't lose any data. In both cases, you'll have holes (missing timesteps) ('date' will have missing 31st, 'year' will have fixed missing dates, see the method's doc).

Depending on what kind of bias-adjustment is being done, sometimes the conversion is better be done in the other direction. For example, if you correct each day-of-year individually, you might prefer removing days from the 365-day data rather that have holes or interpolating the 360-day data.

[ollie-bell]

Yeh I guess the issue with align_on="date" is that in this example we now only have 3582 days, whereas 10 years on a 360_day calendar should be 3600 days. Seems like either way there needs to be some additional modification to the timestamps after the call to convert_calendar.