Allow custom indexes to create new variables

[dcherian]

Is your feature request related to a problem?

Currently it looks like Index.create_variables cannot create a new variable, i.e. it must return a subset of variables passed in set_xindex. Any extra variables returned are ignored.

I am working with a ForecastIndex for weather forecast data cubes that have two dimensions time, step. I'd like the index to return a valid_time = time + step and for ForecastIndex to be associated with time, step, and valid_time.

1. Is there an argument for loosening this restriction?
2. If we don't loosen it, then I think we should raise an error, instead of silently ignoring any extra variables (very confusing).

[dhruvak001]

@dcherian I have opened PR #10503 to address this issue. Please check it out whenever you are free.

[benbovy]

I actually have a similar case where I'd like to add lazy coordinates to a xdggs.DGGSIndex (e.g., the latitude and longitude coordinates of the DGGS cell centroids) in addition to the cell ids coordinate.

Thanks for already submitting a PR @dhruvak001.

I'm wondering, though, if it wouldn't be too magical for set_xindex() to return new coordinates. I understand it as "associate a new index with one or more existing coordinates", so I'd expect it to return an object with updated indexes but I would not expect it to drop or create new coordinates.

A more verbose but probably less confusing alternative may look like:

index = CustomIndex.factory_method(ds, ["time", "step"])
coords = xr.Coordinates.from_xindex(index)
ds.assign_coords(coords)

where ds.assign_coords() already goes through all the variable merging logic and has clear semantics. This approach should already work with no modification to Xarray.

If we don't loosen it, then I think we should raise an error, instead of silently ignoring any extra variables (very confusing).

Agreed!

[dcherian]

Yes, I think we recommend set_xindex only for very simple use cases. I am always needing to parse attributes to figure out what to do with the received variables.

An error pointing out the "factory_method" pattern would be nice.

[dhruvak001]

@benbovy @dcherian Yes, I understand now, this actually would be simple and better implementation. I have updated the PR accordingly. (#10503)