Need data-storage package for 0.7 (JLD no longer working)


#1

I have been using the JLD package for quite some time (since 0.4) to save and checkpoint my data that is shared between Linux and Windows. Right now, neither JLD nor JLD2 is working for Julia 0.7 (Windows). I suppose I could open an issue, but I’m noticing that neither of these packages has been updated for months, so maybe the plan is to abandon JLD and switch to a new data storage format? Can someone advise me about the recommended platform-independent data format for 0.7?

In case any of the package maintainers is reading this, here are the errors I obtained (0.7.0-alpha, downloaded today; packages tried today).

For Pkg.add("JLD") the following error occurred:

┌ Error: Error building `Homebrew`; see log file for further info

By the way, could some helpful reader tell me which log file this message refers to?

Following the Pkg.add statement, I tried Pkg.test("JLD") and got the following error:

┌ Error: Error building `CodecZlib`; see log file for further info

(Same log file?) Then I received hundreds of warnings and this error:

ERROR: LoadError: LoadError: UndefVarError: SimpleVector not defined

Then I tried Pkg.add("JLD2"), which seemed to work, but Pkg.test("JLD2") gave dozens of warnings plus this error:

ERROR: LoadError: LoadError: syntax: local variable name "x" conflicts with an argument

#2

Julia 0.7-alpha has just been tagged. Devs will now begin to support 0.7, but it should take sometime. I would not expect anything before the beta. Finally, I am pretty sure JLD2 will be supported. Not being updated in months can also mean that the package is awesome the way it is :smile:


#3

Yeah, it’s way too early in the v0.7 release to decide that a given package is being abandoned.

In particular, the build issues you’re seeing are with Homebrew.jl and CodeZlib.jl, which are build and test dependencies of JLD, so the first thing to do is to start tracking down the issues in the dependencies. I think the path of the log file is printed in the Pkg3 output if you want to figure out what the full error message is.

The issue you’re seeing with JLD2 looks like a new one. I know for a fact that the authors have been working on v0.7 compatibility, but it’s been a moving target until now. Opening an issue would be the first step towards resolving the problem.


#4

It’s called v0.7-alpha. It’s your job to update JLD/JLD2. If you don’t want to update packages, don’t use the alpha.


#5

Thanks for the suggestions-- I have opened an issue (https://github.com/simonster/JLD2.jl/issues/74) for JLD2. I also tracked down the log file (thanks for the suggestion!) and have opened another issue for Homebrew (https://github.com/JuliaPackaging/Homebrew.jl/issues/233).

Is there an agreed-upon date or sequence of releases when “core” packages are supposed to be ready for 0.7.0? Although JLD is not in the standard library, it is a core piece of functionality for many users. For example, Matlab has provided its load and save commands at least since Matlab 3.5, around 1988, even before Matlab supported sparse matrices.

Also, there is some benefit to the community at large for ordinary users such as myself to be early adopters of new language versions. For example, my code uncovered a performance regression bug in an early version of 0.5 (arrays with >6 subscripts) and two performance regression bugs in early versions of 0.6 (negative of a sparse matrix; concatenating a sparse vector to a sparse matrix).


#6

Maybe ANN: BSON.jl, for saving your Julia data works.


#7

For your matlab example, keep in mind mathworks changed the .mat implementation a few times (inlc. non-backward compatible versions) and today it’s a variant of HDF5. The tricky thing is, julia supports a way broader range of data with composite types.


#8

Certainly — if they open issues, and make PRs. Just complaining about things will not help.

I suppose you should.

Even in a fast-moving package ecosystem, not making any changes for months is not a sign of abandonment. Neither is not updating a few days after an alpha of the next version comes out.

Alpha versions of open source software implicitly carry the expectation that the user is willing to get his/her hands dirty.