Slow File I/O with Hickle

[MarkHolmstrom]

My current workflow requires frequently saving to and loading from disk the replay buffer (AdvancedReplayBuffer), which I noticed was taking a long time both saving and loading as the replay buffer grew in size. Replacing the buffer's save and load calls with the respective Pickle dump and load sped up the I/O significantly and now I am wondering why this is not the framework default.

Overall, Pickle is used for file I/O much more often than Hickle, which is used only for buffers.

[PaParaZz1]

In our previous experiment, hickle is faster than pickle and more suitable for the HDF5 format. Thus we select the library as default. Could you please provide some example data in your case? And we will further investigate this problem.