Pass arguments along with preprocess=... on open_mfdataset

[bradyrx]

I'm looking to pass function arguments (args or kwargs) into the preprocess function from open_mfdataset. Is there any straight-forward way to do this?

E.g.,

# Opening arbitrary set of small NetCDFs for example
ds = xr.open_mfdataset(
    "file_*.nc",
    concat_dim="time",
    preprocess=_slice_models
)

# Arbitrary slicing func. Could use a `NamedTuple` or do any
# other simple, but variable operation.
def _slice_models(ds, x0, x1, y0, y1):
    ds = ds.sel(lon=slice(x0, x1), lat=slice(y0, y1))
    return ds

This would be ideal in the case of reusable code that is loading in large, global high-res datasets and slicing over various sub-domains. Ideally this isn't done with global variables, since the open_mfdataset call will reside in a function itself.

[TomNicholas]

I think I've done this using functools.partial before. Using that you can bind any args or kwargs into the function before passing it in to open_mfdataset, but still have it wait upon the ds argument. This pattern might be a good addition to a "recipes" page or similar (or just an example in the open_mfdataset docstring).

[bradyrx]

Great suggestion @TomNicholas, thanks! I’ve used that elsewhere for generalizing nodes in a data pipeline. I’ll give that a try tomorrow at work and if it works out cleanly I’ll add it to the docs.

[bradyrx]

@andersy005 , @TomNicholas , @dcherian , this worked like a charm. Added a brief docstring example here: #6825.