Can't read old JLD2 file

[Juan]

What should we use then?
JLD2, HDF5, Feather, Parquet, JLD, fst…?

Some people say JLD produce memory leakage.

The poor state of fileformats for High Performance computing

About two weeks ago I have put a post asking for recommending the file-format use with high-perfomance computing. Our use-case is training of relatively simple neural networks but using terabytes of data. We do that on a single machine with GPU, where there are separate processes loading the data from S3 storage and preparing them for the master thread. After our reasearch, we have come with following alternatives: Feather Parquet ProtoBuf JLD JLD2 HDF5 Let’s discuss all options. Feathe…

It would be great to have something like TileDB. Or something fast, escalable and able to add rows and columns on disk.