BUG: Large tensorstore already-exists failure #155
Merged
+254
−160
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thewtex
force-pushed
the
tensorstore-crash
branch
from
e91fff9 to
9d7d238
Compare
thewtex
changed the title
When writing large arrays in regions, the code calls _write_array_with_tensorstore multiple times (once for each region), and each time it tries to create a new TensorStore dataset with create=True. After the first region is written, the dataset already exists, so subsequent calls fail with "ALREADY_EXISTS". Modify the _write_with_tensorstore function to handle the case where the dataset already exists. Check if it's the first region being written (or create the dataset only once) and then open existing datasets for subsequent regions. Re: #152
thewtex
force-pushed
the
tensorstore-crash
branch
from
9d7d238 to
133de45
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Re: #152