Skip to content

Remove breaking check about auto_mkdir for FSSpecStore #3193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Jul 3, 2025
Merged

Remove breaking check about auto_mkdir for FSSpecStore #3193

merged 7 commits into from
Jul 3, 2025

Conversation

maxrjones
Copy link
Member

@maxrjones maxrjones commented Jul 2, 2025 •
edited
Loading

This change is causing downstream breakages since auto_mkdir is only relevant for stores with read_only = False and stores that are in the root of the provided directory.

E.g., https://github.com/developmentseed/titiler/actions/runs/16025154840/job/45211163911#step:8:2093

This fixes a regression introduced in #2774. I apologize for the mistake in that pull request.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.rst
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 2, 2025
@maxrjones maxrjones changed the title Remove breaking check from _make_async Remove breaking check about auto_mkdir for FSSpecStore Jul 2, 2025
@maxrjones maxrjones mentioned this pull request Jul 2, 2025
Merged
6 tasks
@d-v-b d-v-b added this to the 3.0.10 milestone Jul 3, 2025
@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 3, 2025
d-v-b
d-v-b approved these changes Jul 3, 2025
View reviewed changes
Copy link
Contributor

@d-v-b d-v-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed some changes in the interest of speed. thanks for this quick fix max!

@d-v-b d-v-b merged commit 5a24487 into zarr-developers:main Jul 3, 2025
30 checks passed
@d-v-b
Copy link
Contributor

d-v-b commented Jul 3, 2025

@meeseeksdev backport to 3.0.10

@lumberbot-app Lumberbot (App)
Copy link

lumberbot-app bot commented Jul 3, 2025

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 3.0.10
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 5a24487f09d499c91ce25f24af910c2ede055b5a
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #3193: Remove breaking check about `auto_mkdir` for FSSpecStore'
  1. Push to a named branch:
git push YOURFORK 3.0.10:auto-backport-of-pr-3193-on-3.0.10
  1. Create a PR against branch 3.0.10, I would have named this PR:

"Backport PR #3193 on branch 3.0.10 (Remove breaking check about auto_mkdir for FSSpecStore)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

d-v-b added a commit to d-v-b/zarr-python that referenced this pull request Jul 3, 2025
@maxrjones @d-v-b
Remove breaking check about auto_mkdir for FSSpecStore (zarr-develo…
8d51dd3 
…pers#3193)

* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
(cherry picked from commit 5a24487)
d-v-b added a commit that referenced this pull request Jul 3, 2025
@d-v-b @maxrjones
Remove breaking check about auto_mkdir for FSSpecStore (#3193) (#3203)
1e66c40 
* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------


(cherry picked from commit 5a24487)

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
d-v-b added a commit that referenced this pull request Jul 3, 2025
@d-v-b @maxrjones @tasansal
Auto backport of pr 3195 on 3.0.10 (#3204)
e0ba359 
* Remove breaking check about `auto_mkdir` for FSSpecStore (#3193)

* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
(cherry picked from commit 5a24487)

* Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195)

* Add missing import for AsyncFileSystemWrapper in `_fsspec.py`

* Add missing changelog entry for AsyncFileSystemWrapper import fix

* Move AsyncFileSystemWrapper import past the version check in `_fsspec.py`

* Add newline after AsyncFileSystemWrapper import in `_fsspec.py`

* Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py`

---------

Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
(cherry picked from commit 97aa42f)

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com>
Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
@maxrjones maxrjones mentioned this pull request Jul 6, 2025
Open
d-v-b added a commit that referenced this pull request Jul 9, 2025
@dstansby @meeseeksmachine
@lukasbindreiter @maxrjones @d-v-b @TomNicholas @tasansal
Merge 3.0.9 & 3.0.10 into main (#3220)
23edb80 
* Enable tests on 3.0.x branch (#3135)

* Changelog for 3.0.9

* Backport PR #3149: Add GroupNotFound error to API docs (#3179)

Co-authored-by: David Stansby <dstansby@gmail.com>

* Backport PR #3140: Suppress FileNotFoundError when deleting keys in the obstore adapter (#3180)

Co-authored-by: Lukas Bindreiter <lukas.bindreiter@tilebox.com>

* Backport PR #3138: Add with_read_only() convenience method to store (#3181)

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

* Create read only copy if needed when opening a store path (#3156)

* Create read only copy if needed when opening a store path

* Add ValueError to Raises section

* Update expected warning

* Update src/zarr/storage/_common.py

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

* Use ANY_ACCESS_MODE

* Update src/zarr/storage/_common.py

Co-authored-by: David Stansby <dstansby@gmail.com>

* Update src/zarr/storage/_common.py

Co-authored-by: David Stansby <dstansby@gmail.com>

* Update changes

* Try using get_args on definition

* Revert "Try using get_args on definition"

This reverts commit 7ad760f.

* Add test

* Remove warning

* Apply suggestion for try; except shortening

Co-authored-by: Tom Nicholas <tom@earthmover.io>

* Improve code coverage

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: David Stansby <dstansby@gmail.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>
(cherry picked from commit 5731c6c)

* Create read only copy if needed when opening a store path (#3156) (#3182)

* Create read only copy if needed when opening a store path

* Add ValueError to Raises section

* Update expected warning

* Update src/zarr/storage/_common.py



* Use ANY_ACCESS_MODE

* Update src/zarr/storage/_common.py



* Update src/zarr/storage/_common.py



* Update changes

* Try using get_args on definition

* Revert "Try using get_args on definition"

This reverts commit 7ad760f.

* Add test

* Remove warning

* Apply suggestion for try; except shortening



* Improve code coverage

---------




(cherry picked from commit 5731c6c)

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
Co-authored-by: David Stansby <dstansby@gmail.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>

* Remove breaking check about `auto_mkdir` for FSSpecStore (#3193)

* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
(cherry picked from commit 5a24487)

* Remove breaking check about `auto_mkdir` for FSSpecStore (#3193) (#3203)

* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------


(cherry picked from commit 5a24487)

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

* Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195)

* Add missing import for AsyncFileSystemWrapper in `_fsspec.py`

* Add missing changelog entry for AsyncFileSystemWrapper import fix

* Move AsyncFileSystemWrapper import past the version check in `_fsspec.py`

* Add newline after AsyncFileSystemWrapper import in `_fsspec.py`

* Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py`

---------

Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
(cherry picked from commit 97aa42f)

* Auto backport of pr 3195 on 3.0.10 (#3204)

* Remove breaking check about `auto_mkdir` for FSSpecStore (#3193)

* Remove breaking check from _make_async

* Update expected error

* Change import structure to protect against AttributeError

* changelog

* add test to ensure that we can create a read-only copy of the store with auto_mkdir=False

* only test if the async wrapper is available

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
(cherry picked from commit 5a24487)

* Add missing import for AsyncFileSystemWrapper for `_make_async` in `_fsspec.py` (#3195)

* Add missing import for AsyncFileSystemWrapper in `_fsspec.py`

* Add missing changelog entry for AsyncFileSystemWrapper import fix

* Move AsyncFileSystemWrapper import past the version check in `_fsspec.py`

* Add newline after AsyncFileSystemWrapper import in `_fsspec.py`

* Simplify import statement for AsyncFileSystemWrapper in `_fsspec.py`

---------

Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
(cherry picked from commit 97aa42f)

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com>
Co-authored-by: Altay Sansal <altay.sansal@tgs.com>

* 3.0.9 release notes (#3183)

* Create read only copy if needed when opening a store path (#3156)

* Create read only copy if needed when opening a store path

* Add ValueError to Raises section

* Update expected warning

* Update src/zarr/storage/_common.py

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

* Use ANY_ACCESS_MODE

* Update src/zarr/storage/_common.py

Co-authored-by: David Stansby <dstansby@gmail.com>

* Update src/zarr/storage/_common.py

Co-authored-by: David Stansby <dstansby@gmail.com>

* Update changes

* Try using get_args on definition

* Revert "Try using get_args on definition"

This reverts commit 7ad760f.

* Add test

* Remove warning

* Apply suggestion for try; except shortening

Co-authored-by: Tom Nicholas <tom@earthmover.io>

* Improve code coverage

---------

Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: David Stansby <dstansby@gmail.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>
(cherry picked from commit 5731c6c)

* release notes

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
Co-authored-by: David Stansby <dstansby@gmail.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>

* release notes

---------

Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com>
Co-authored-by: Lukas Bindreiter <lukas.bindreiter@tilebox.com>
Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Tom Nicholas <tom@earthmover.io>
Co-authored-by: Altay Sansal <tasansal@users.noreply.github.com>
Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@d-v-b d-v-b d-v-b approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.0.10
Development

Successfully merging this pull request may close these issues.
2 participants
@maxrjones @d-v-b