Status of libraries for saving binary files: composite datatypes

General Usage
, ,
1

Hi,

I am having some serious trouble saving composite datatypes.

For small output files, BSON works fine. However, in cases where it is hard to split the output into many chunks, I tried to use JLD2. Unfortunately, this comes with all sort of issues. The discussion in Can't read old JLD2 file - #16 by Tomas_Pevny and UnsupportedVersionException ?!?! · Issue #140 · JuliaIO/JLD2.jl · GitHub are still very relevant I am afraid.

I was wondering what is the current status of development for JLD2 (and similar libraries). I understand that it is quite hard to structure efficient routines to save composite datatypes, but it would also be particularly convenient in many applications. If this is not possible, would it be a good idea to have a wrapper for HDF5 to convert composite datatypes into groups (while saving the output file)?

2

Well, I always wanted to use CBOR for this, so I contributed to it in the pre-1.0 era, but I’m not sure if it was fully updated to Julia v1 yet:

2 Likes
3

I did not know about this package. It seems like a good alternative to JLD2. However, I wonder whether I can expect long term support for the package.