Merge branch 'main' of https://github.com/zarr-developers/zarr-python

Author: d-v-b

.github/workflows/needs_release_notes.yml

@@ -5,6 +5,7 @@ on:
 
 jobs:
   triage:
+    if: ${{ github.event.pull_request.user.login != 'dependabot[bot]' }} && ${{ github.event.pull_request.user.login != 'pre-commit-ci[bot]' }}
     runs-on: ubuntu-latest
     steps:
     - uses: actions/labeler@main

docs/release.rst

@@ -17,6 +17,8 @@ Unreleased
 * Fix bug that caused double counting of groups in ``groups()`` and ``group_keys()``
   methods with V3 stores.
   By :user:`Ryan Abernathey <rabernat>` :issue:`1228`.
+* Handle fsspec.FSMap using FSStore store
+  By :user:`Rafal Wojdyla <ravwojdyla>` :issue:`1304`.
 
 * Add support for setting user-defined attributes at array / group creation time.
   By :user: `Davis Bennett <d-v-b>` :issue:`538`.

docs/spec/v1.rst

@@ -144,9 +144,9 @@ are converted to strings and concatenated with the period character
 ('.') separating each index. For example, given an array with shape
 (10000, 10000) and chunk shape (1000, 1000) there will be 100 chunks
 laid out in a 10 by 10 grid. The chunk with indices (0, 0) provides
-data for rows 0-1000 and columns 0-1000 and is stored under the key
-'0.0'; the chunk with indices (2, 4) provides data for rows 2000-3000
-and columns 4000-5000 and is stored under the key '2.4'; etc.
+data for rows 0-999 and columns 0-999 and is stored under the key
+'0.0'; the chunk with indices (2, 4) provides data for rows 2000-2999
+and columns 4000-4999 and is stored under the key '2.4'; etc.
 
 There is no need for all chunks to be present within an array
 store. If a chunk is not present then it is considered to be in an

docs/spec/v2.rst

@@ -216,9 +216,9 @@ To form a string key for a chunk, the indices are converted to strings and
 concatenated with the period character (".") separating each index. For
 example, given an array with shape (10000, 10000) and chunk shape (1000, 1000)
 there will be 100 chunks laid out in a 10 by 10 grid. The chunk with indices
-(0, 0) provides data for rows 0-1000 and columns 0-1000 and is stored under the
-key "0.0"; the chunk with indices (2, 4) provides data for rows 2000-3000 and
-columns 4000-5000 and is stored under the key "2.4"; etc.
+(0, 0) provides data for rows 0-999 and columns 0-999 and is stored under the
+key "0.0"; the chunk with indices (2, 4) provides data for rows 2000-2999 and
+columns 4000-4999 and is stored under the key "2.4"; etc.
 
 There is no need for all chunks to be present within an array store. If a chunk
 is not present then it is considered to be in an uninitialized state.  An

docs/tutorial.rst

@@ -525,9 +525,9 @@ When the indexing arrays have different shapes, they are broadcast together.
 That is, the following two calls are equivalent::
 
     >>> z[1, [1, 3]]
-    array([5, 7])
+    array([6, 8])
     >>> z[[1, 1], [1, 3]]
-    array([5, 7])
+    array([6, 8])
 
 Indexing with a mask array
 ~~~~~~~~~~~~~~~~~~~~~~~~~~

requirements_dev_minimal.txt

@@ -3,6 +3,6 @@ asciitree==0.3.3
 fasteners==0.18
 numcodecs==0.11.0
 msgpack-python==0.5.6
-setuptools-scm==7.0.5
+setuptools-scm==7.1.0
 # test requirements
 pytest==7.2.0

requirements_dev_numpy.txt

@@ -1,4 +1,4 @@
 # Break this out into a separate file to allow testing against
 # different versions of numpy. This file should pin to the latest
 # numpy version.
-numpy==1.23.5
+numpy==1.24.0

requirements_dev_optional.txt

@@ -1,9 +1,9 @@
 # optional library requirements
 # bsddb3==6.2.6; sys_platform != 'win32'
-lmdb==1.3.0; sys_platform != 'win32'
+lmdb==1.4.0; sys_platform != 'win32'
 # optional library requirements for Jupyter
 ipytree==0.2.2
-ipywidgets==8.0.2
+ipywidgets==8.0.3
 # optional library requirements for services
 # don't let pyup change pinning for azure-storage-blob, need to pin to older
 # version to get compatibility with azure storage emulator on appveyor (FIXME)
@@ -14,7 +14,6 @@ types-setuptools
 pymongo==4.3.3
 # optional test requirements
 coverage
-flake8==5.0.4
 pytest-cov==4.0.0
 pytest-doctestplus==0.12.1
 pytest-timeout==2.1.0

zarr/_storage/v3.py

@@ -567,6 +567,16 @@ def _normalize_store_arg_v3(store: Any, storage_options=None, mode="r") -> BaseS
         return store
     if isinstance(store, os.PathLike):
         store = os.fspath(store)
+    if FSStore._fsspec_installed():
+        import fsspec
+        if isinstance(store, fsspec.FSMap):
+            return FSStoreV3(store.root,
+                             fs=store.fs,
+                             mode=mode,
+                             check=store.check,
+                             create=store.create,
+                             missing_exceptions=store.missing_exceptions,
+                             **(storage_options or {}))
     if isinstance(store, str):
         if "://" in store or "::" in store:
             store = FSStoreV3(store, mode=mode, **(storage_options or {}))

zarr/storage.py

@@ -142,6 +142,16 @@ def _normalize_store_arg_v2(store: Any, storage_options=None, mode="r") -> BaseS
         return store
     if isinstance(store, os.PathLike):
         store = os.fspath(store)
+    if FSStore._fsspec_installed():
+        import fsspec
+        if isinstance(store, fsspec.FSMap):
+            return FSStore(store.root,
+                           fs=store.fs,
+                           mode=mode,
+                           check=store.check,
+                           create=store.create,
+                           missing_exceptions=store.missing_exceptions,
+                           **(storage_options or {}))
     if isinstance(store, str):
         if "://" in store or "::" in store:
             return FSStore(store, mode=mode, **(storage_options or {}))
@@ -1341,6 +1351,8 @@ def __init__(self, url, normalize_keys=False, key_separator=None,
                  create=False,
                  missing_exceptions=None,
                  **storage_options):
+        if not self._fsspec_installed():  # pragma: no cover
+            raise ImportError("`fsspec` is required to use zarr's FSStore")
         import fsspec
 
         mapper_options = {"check": check, "create": create}
@@ -1512,6 +1524,13 @@ def clear(self):
             raise ReadOnlyError()
         self.map.clear()
 
+    @classmethod
+    def _fsspec_installed(cls):
+        """Returns true if fsspec is installed"""
+        import importlib.util
+
+        return importlib.util.find_spec("fsspec") is not None
+
 
 class TempStore(DirectoryStore):
     """Directory store using a temporary directory for storage.