Is it possible to overwrite a dataset "in place" with Icechunk? #837
-
|
Every so often I have to add/insert some additional information to my datasets, the only inconvenient is that I always try to keep the labels in a sorted order and this extra information can break that condition, and when that happens I have to rewrite the whole array into a temporal path and move it to the original path (2 operations). My main question is if this process could be improved using Icechunk capabilities, from my understanding Icechunk preserves the commit history, so if I try to overwrite the dataset it should not suffer the same issue of reading and writing on the same location at the same time as regular Xarray + Zarr solution, is my assumption correct? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Yes correct. Icechunk's transactional design means that this operation is safe. |
Beta Was this translation helpful? Give feedback.
Yes correct. Icechunk's transactional design means that this operation is safe.