Skip to content

Commit 5b6d757

Browse files
Zarr: drop "source" and "original_shape" from encoding (#7500)
* Implement dropping of encoding added by xarray in 'to_zarr' * add test xarray added encoding is dropped in 'to_zarr' * add changes to whats new * Fix typo Co-authored-by: Joe Hamman <jhamman1@gmail.com> --------- Co-authored-by: Joe Hamman <jhamman1@gmail.com>
1 parent bc71591 commit 5b6d757

File tree

3 files changed

+14
-0
lines changed

3 files changed

+14
-0
lines changed

doc/whats-new.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,9 @@ Bug fixes
5858
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_ and `Scott Chamberlin <https://github.com/scottcha>`_.
5959
- Handle ``keep_attrs`` option in binary operators of :py:meth:`Dataset` (:issue:`7390`, :pull:`7391`).
6060
By `Aron Gergely <https://github.com/arongergely>`_.
61+
- :py:func:`xarray.Dataset.to_zarr` now drops variable encodings that have been added by xarray during reading
62+
a dataset. (:issue:`7129`, :pull:`7500`).
63+
By `Hauke Schulz <https://github.com/observingClouds>`_.
6164

6265
Documentation
6366
~~~~~~~~~~~~~

xarray/backends/zarr.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,7 @@ def extract_zarr_variable_encoding(
230230
"""
231231
encoding = variable.encoding.copy()
232232

233+
safe_to_drop = {"source", "original_shape"}
233234
valid_encodings = {
234235
"chunks",
235236
"compressor",
@@ -238,6 +239,10 @@ def extract_zarr_variable_encoding(
238239
"write_empty_chunks",
239240
}
240241

242+
for k in safe_to_drop:
243+
if k in encoding:
244+
del encoding[k]
245+
241246
if raise_on_invalid:
242247
invalid = [k for k in encoding if k not in valid_encodings]
243248
if invalid:

xarray/tests/test_backends.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2036,6 +2036,12 @@ def test_chunk_encoding_with_dask(self) -> None:
20362036
# don't actually check equality because the data could be corrupted
20372037
pass
20382038

2039+
def test_drop_encoding(self):
2040+
ds = open_example_dataset("example_1.nc")
2041+
encodings = {v: {**ds[v].encoding} for v in ds.data_vars}
2042+
with self.create_zarr_target() as store:
2043+
ds.to_zarr(store, encoding=encodings)
2044+
20392045
def test_hidden_zarr_keys(self) -> None:
20402046
expected = create_test_data()
20412047
with self.create_store() as store:

0 commit comments

Comments
 (0)