Option to cache file in `open_virtual_dataset`

[TomNicholas]

#557 relies on the ability to pass cache=True, through the un-merged (and since closed) PR #564. We need another way to do this. Maybe it's now already possible? In that case it needs documenting, probably as part of #590.

cc @maxrjones

[maxrjones]

It's not currently supported. @chuckwondo suggested in #568 (comment) that we could allow a user to pass in their own store rather than their own reader, and that store could be a "caching" store. I started https://github.com/virtual-zarr/obspec-utils to experiment with different stores build on obspec, with an initial trial of a caching store.

[TomNicholas]

we could allow a user to pass in their own store rather than their own reader, and that store could be a "caching" store

So IIUC we have this now, we just need the CachingStore.

I started https://github.com/virtual-zarr/obspec-utils to experiment with different stores build on obspec, with an initial trial of a caching store.

Cool - I don't see any CachingStore in there (or any code at all) - did you forget to push something or just haven't got around to it?

[maxrjones]

we could allow a user to pass in their own store rather than their own reader, and that store could be a "caching" store

So IIUC we have this now, we just need the CachingStore.

yes!

I started virtual-zarr/obspec-utils to experiment with different stores build on obspec, with an initial trial of a caching store.

Cool - I don't see any CachingStore in there (or any code at all) - did you forget to push something or just haven't got around to it?

I had reached the limit of productive procrastination before getting to some real deadlines so there is no actual code there yet 😓 I'd encourage others to take up the implementation.