General file storage in ice chunk

[Leonard-Mueller]

Hi,
I wanted to ask if it is also possible to store general files in icechunk or if a feature like that is planned. I saw that some work has been done for other raster file formats via virtualzarr.
Some background:
In our team we currently are looking for a solution to implement a version control system for our datasets. The vast majority of it is raster data, which would be easily migratable. Some of the data though is in other file formats like csv, or shape files. These files are not large by any means, so chunking is not a requirement. It would be nice to be able to version control them the same way as our main datasets as they are required in the same pipelines.

Has something like this been implemented? I wasn't able to find anything like this in the documentation so far.

Some hacky idea I came up with was serializing the files and storing them as byte arrays in native zarr. Though this feels like a future accident to happen.

[paraseba]

We don't have plans to implement this at the moment. But it sounds like something a lightweight library on top of Icechunk could do.