Problem saving SharedArrays with JLD type

General Usage
1

I’ve got the following issue. I generate data using the Distributed module, storing it in a SharedArray, and then dumping it to disk via FileIO. The problem I have is that if I save it as .jld when I try to load back in, I am getting the following error:

┌ Warning: type Distributed.RRID not present in workspace; reconstructing
└ @ JLD /Users/guardian/.julia/packages/JLD/jeGJb/src/jld_types.jl:697
Error encountered while loading "/Users/guardian/.myfile.jld".

Fatal error:

stored type SharedArrays.SharedArray{Core.Float64,2} does not match currently loaded type

Stacktrace:
 [1] handle_error(::JLD.TypeMismatchException, ::File{DataFormat{:JLD}}) at /Users/guardian/.julia/packages/FileIO/zL0JE/src/error_handling.jl:82
 [2] handle_exceptions(::Array{Any,1}, ::String) at /Users/guardian/.julia/packages/FileIO/zL0JE/src/error_handling.jl:77
 [3] load(::FileIO.Formatted; options::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}) at /Users/guardian/.julia/packages/FileIO/zL0JE/src/loadsave.jl:189
 [4] load at /Users/guardian/.julia/packages/FileIO/zL0JE/src/loadsave.jl:166 [inlined]
 [5] #load#13 at /Users/guardian/.julia/packages/FileIO/zL0JE/src/loadsave.jl:118 [inlined]
 [6] load(::String) at /Users/guardian/.julia/packages/FileIO/zL0JE/src/loadsave.jl:118
 [7] top-level scope at In[2]:8
 [8] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091

In contrast, if I save as .jld2 (which I know is deprecated/abandoned), there is no issue. How can I correct this in the JLD format so as to maintain longterm support?

1 Like
2

My current understanding is that JLD2 is being actively maintained again despite the warning on the repo. I hope so because I moved some code from JLD to JLD2 quite some time ago and do not wish to make another change.

3

I always saved SharedArrays using JLD and it (suddenly) gave me the same error.
Uncommenting lines 723-725 of jld_types.jl solved it for me.

I.e. uncomment these lines:

# if T !== Expr  # in julia >= 0.7 Expr has 1 fewer fields; the trailing field can be ignored.
#     dtype == newtype || throw(TypeMismatchException(typename))
# end

I tried to print out dtype and newtype and they are identical, I don’t know why it thinks they are not…

4

Actually this #281 solves it. Just need to upgrade HDF5 to v13.7 or downgrade HDF5_jll to v1.10.5