Need data-storage package for 0.7 (JLD no longer working)

Thanks for the suggestion. In fact I tried this already, and

pkg> add https://github.com/JeffBezanson/JLD2.jl#gdkrmr/0.7compat

worked fine for me to install it directly.

That said, my experience with JLD2.jl and BSON.jl made me rethink my approach to long-term data storage using types directly from Julia.

First, I don’t think it is really suitable for long-term archiving of binary data with nontrivial types in the sense that HDF5 is designed for: reading becomes tricky when the representation changes, which is bound to happen. Then the advantage compared to serialize is not that clear. Because of this, now I think I should stick to plain vanilla HDF5 and think a bit more about converting my data to the types it supports.

Second, even if the first problem had a solution, I have concerns about the long-term support of these packages. With the migration to v0.7, it again became evident that packages maintained by a single person (or at most a few people) have a potentially very limited lifecycle: the original author may move on after a few months of interest, neglecting or even abandonning the package. This is in principle not a problem with most packages as one can fork and continue, but with packages like JLD2, this can be difficult. I am not claiming that JLD2 is abandonned (as far as I know, the maintaner just could be on a vacation, etc), but if that should happen, recovering old data a few years from now on would be a labor-intensive exercise.

3 Likes