Skip to content

Commit 97887fd

Browse files
jaichermax-sixty
andauthored
Fix saving chunked datasets with zero length dimensions (#5742)
* Added test_save_emptydim for zarr backends, which fails when chunking Added test that fails when saving to zarr a dataset with a chunked array that has a dimension of length zero (Issue #5741) * Load all variables with zero entries before saving to_zarr This addresses Issue #5741 and allows `test_save_emptydim` to pass. We get around `to_zarr` not liking dask arrays with zero length dimensions by giving it numpy arrays, which works for some reason * Updated whats-new.rst with information about fix for #5741 Co-authored-by: Maximilian Roos <5635139+max-sixty@users.noreply.github.com>
1 parent fd42449 commit 97887fd

File tree

3 files changed

+18
-0
lines changed

3 files changed

+18
-0
lines changed

doc/whats-new.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,9 @@ Deprecations
6868

6969
Bug fixes
7070
~~~~~~~~~
71+
72+
- Fix ZeroDivisionError from saving dask array with empty dimension (:issue: `5741`).
73+
By `Joseph K Aicher <https://github.com/jaicher>`_.
7174
- Fixed performance bug where ``cftime`` import attempted within various core operations if ``cftime`` not
7275
installed (:pull:`5640`).
7376
By `Luke Sewell <https://github.com/lusewell>`_

xarray/backends/api.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1322,6 +1322,11 @@ def to_zarr(
13221322
See `Dataset.to_zarr` for full API docs.
13231323
"""
13241324

1325+
# Load empty arrays to avoid bug saving zero length dimensions (Issue #5741)
1326+
for v in dataset.variables.values():
1327+
if v.size == 0:
1328+
v.load()
1329+
13251330
# expand str and Path arguments
13261331
store = _normalize_path(store)
13271332
chunk_store = _normalize_path(chunk_store)

xarray/tests/test_backends.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2186,6 +2186,16 @@ def test_to_zarr_append_compute_false_roundtrip(self):
21862186
with self.open(store) as actual:
21872187
assert_identical(xr.concat([ds, ds_to_append], dim="time"), actual)
21882188

2189+
@pytest.mark.parametrize("chunk", [False, True])
2190+
def test_save_emptydim(self, chunk):
2191+
if chunk and not has_dask:
2192+
pytest.skip("requires dask")
2193+
ds = Dataset({"x": (("a", "b"), np.empty((5, 0))), "y": ("a", [1, 2, 5, 8, 9])})
2194+
if chunk:
2195+
ds = ds.chunk({}) # chunk dataset to save dask array
2196+
with self.roundtrip(ds) as ds_reload:
2197+
assert_identical(ds, ds_reload)
2198+
21892199
@pytest.mark.parametrize("consolidated", [False, True])
21902200
@pytest.mark.parametrize("compute", [False, True])
21912201
@pytest.mark.parametrize("use_dask", [False, True])

0 commit comments

Comments
 (0)