recommended method to update variable values

[JessicaS11]

My understanding is that xr.update() is meant to be deprecated (as a side note, I was surprised not to get a deprecation warning when I tried to use it in version 2022.3.0). I'm wondering what the recommended pathway for updating variable values is without losing their coordinate information.

MWE:

temp = 15 + 8 * np.random.randn(2, 2, 1)

lon = [[-99.83, -99.32], [-99.79, -99.23]]

lat = [[42.25, 42.21], [42.63, 42.59]]

# for real use cases, its good practice to supply array attributes such as
# units, but we won't bother here for the sake of brevity
ds = xr.Dataset(
    {
        "temperature": (["x", "y", "time_idx"], temp),
        "time": ("time_idx", [b'2019-01-11T05:26:31.323722Z'])
    },
    coords={
        "lon": (["x", "y"], lon),
        "lat": (["x", "y"], lat),
        "time_idx": [15],
    },
)
ds

returns:

In order to convert my time variable to np.datetime, I need to remove the 'Z'. I can easily do the conversion with np.datetime64(ds["time"].data[i].astype(str)[:-1]).

However, I cannot replace the existing ds.time value with this retyped (and slightly modified) value without losing the indexing. Specifically, ds["time"] = np.datetime64(ds["time"].data[0].astype(str)[:-1]) results in:

How can I update variable values while preserving the coordinate references? Losing them is resulting in merge conflicts later in my pipeline. Thanks!

EDIT: I want to clarify that I am trying to do this conversion generically (e.g. where keyword="time"), so the coordinates of a variable being converted are not necessarily known to specify manually, I just want to retain the ones that are already there by supplying a new array of the proper shape and size.

[keewis]

you should be able to use copy(data=...):

In [9]: ds["time"] = ds["time"].copy(data=[np.datetime64(ds["time"].data[0].astype(str)[:-1])])
   ...: ds
Out[9]: 
<xarray.Dataset>
Dimensions:      (x: 2, y: 2, time_idx: 1)
Coordinates:
    lon          (x, y) float64 -99.83 -99.32 -99.79 -99.23
    lat          (x, y) float64 42.25 42.21 42.63 42.59
  * time_idx     (time_idx) int64 15
Dimensions without coordinates: x, y
Data variables:
    temperature  (x, y, time_idx) float64 26.84 -2.025 17.72 3.238
    time         (time_idx) datetime64[ns] 2019-01-11T05:26:31.323722

but do note that while I get a segfault with ds["time"].astype(np.datetime64), ds["time"].astype(str).astype(np.datetime64) seems to work

edit: if you don't want to modify in-place, you can also use .assign:

In [7]: ds.assign({"time": lambda ds_: ds_["time"].astype(str).astype(np.datetime64)})
Out[7]: 
<xarray.Dataset>
Dimensions:      (x: 2, y: 2, time_idx: 1)
Coordinates:
    lon          (x, y) float64 -99.83 -99.32 -99.79 -99.23
    lat          (x, y) float64 42.25 42.21 42.63 42.59
  * time_idx     (time_idx) int64 15
Dimensions without coordinates: x, y
Data variables:
    temperature  (x, y, time_idx) float64 14.82 27.88 16.31 3.407
    time         (time_idx) datetime64[ns] 2019-01-11T05:26:31.323722

[keewis]

the segfault is most likely a bug in numpy, but I also get a deprecation warning if I try ds["time"].data.astype(str).astype(np.datetime64) in pure numpy.

Instead, we can use the .str accessor:

In [12]: ds["time"].str[:-1].astype(np.datetime64)
Out[12]: 
<xarray.DataArray 'time' (time_idx: 1)>
array(['2019-01-11T05:26:31.323722000'], dtype='datetime64[ns]')
Coordinates:
  * time_idx  (time_idx) int64 15

[JessicaS11]

Thanks for all these great suggestions! They're all working beautifully.

the segfault is most likely a bug in numpy

Exactly - the segfault is a non-surfaced post-deprecation error from numpy because numpy datetime no longer automatically handles time zones (hence needing to manually remove the 'Z' from the string; we're trying to get the data producer to update!).

[max-sixty]

+1 for .assign. It also goes nicely in a fluent pipeline; i.e. ds.assign(...).sum(...).assign etc

[keewis]

regarding the deprecation: the update method itself is not deprecated, but it will return None instead of self in the future. Raising deprecation warnings for the return value is really hard, so we only mention that in the docs.

[JessicaS11]

That makes sense - thanks for the clarification!