How does xarray's file caching work exactly (when datasets are loaded with dask)?

[pjpetersik]

I encountered an unexpected behaviour regarding xarray datasets which are loaded into dask arrays.

When I open a dataset with chunks="auto" (load datasets into dask array), then delete the underlying NetCDF file and then execute .load(), this does not lead to any erros and I can still access the values of the arrays.

Here is a minimal example:

import xarray as xr
import os
from pathlib import Path
import dask

# create a NetCDF file which would not even fit into my memory
ds = xr.DataArray(dask.array.ones((50000, 50000)), name="variable")
path = Path("/tmp/saved_on_disk.nc")
ds.to_netcdf(path)

# reopen the NetCDF file and load the dataset with dask
ds_reopen = xr.open_dataset(path, chunks="auto")

# remove the NetCDF file from disk
os.remove(path)

print(ds_reopen.variable.isel(dim_0=0, dim_1=0).load())
# <xarray.DataArray 'variable' ()>
# array(1.)

I always thought that dask loads data from the disk only when needed but here the respective file on the disk is removed.

So the question is, where does the data come from when I execute ds_reopen.load()? I have the feeling that this behaviour might be connected to the CachingFileManager as @shoyer describes in #4240. Is xarray generating some sort of internal copy of the file when opening it?