Skip to content

Conversation

@thewtex
Copy link
Collaborator

@thewtex thewtex commented Jul 7, 2025

Different regions of an array could previously try to specify different
chunk dimensions. Use only the chunking of the region when creating the
dataset.

Fixes #161

Different regions of an array could previously try to specify different
chunk dimensions. Use only the chunking of the region when creating the
dataset.

Fixes #161
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR ensures consistent chunk metadata when using TensorStore, refactors TensorStore spec creation, and adds comprehensive tests for chunk consistency and zero-size validation.

  • Added three pytest cases to verify TensorStore chunk shape consistency, sharding behavior, and zero‐chunk handling.
  • Refactored _write_with_tensorstore to only include chunk metadata on initial dataset creation, build a minimal spec for existing stores, and retry writes on dimension/shape errors.
  • Updated sharding logic in to_ngff_zarr and related helpers to standardize shard/chunk parameters and clean up sharding kwargs.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
py/test/test_to_ngff_zarr_tensorstore.py New tests for TensorStore chunk shape consistency and zero‐chunk validation
py/ngff_zarr/to_ngff_zarr.py Refactored TensorStore writing spec, exception handling, shard/chunk config
Comments suppressed due to low confidence (1)

py/test/test_to_ngff_zarr_tensorstore.py:149

  • The skip reason mentions 3.0.0b1 but the version check is against 3.0.8. Update the reason to accurately reflect the required minimum version.
    zarr_version < version.parse("3.0.8"), reason="zarr version < 3.0.0b1"

@thewtex thewtex force-pushed the tensorstore-chunk-consist-rebase branch from b2b9351 to f8817ba Compare July 7, 2025 18:37
@thewtex thewtex merged commit 32bd137 into main Jul 7, 2025
93 checks passed
@thewtex thewtex deleted the tensorstore-chunk-consist-rebase branch July 7, 2025 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FAILED_PRECONDITION on tensorstore for certain starting image shapes

2 participants