I feel like I am over complicating this - HDF5 saving array

Hello guys

I want to save an array of the following type;

Array{Array{Float32,2},1}

I have called it “B” and I want to save it in a hdf5 file. It gives me an error, with no method matching and I don’t know how to get past it. It works if I try to save only a part of B, ie. B[1], but does not work if I try to save the whole array directly. I am pretty sure I am misunderstanding something very simple, so I hope someone could push me in the right direction?

Kind regards

using HDF5

 B = Vector{Array{Float32,2}}(undef,3)
 B[1] = rand(3,3);
 B[2] = rand(3,3);
 B[3] = rand(3,3);

h5open("B.hdf5", "w") do file
    write(file, "B",B)  # alternatively, say "@write file A"
    close(file)
end

Is HDF5 a requirement? If not then you can use JLD.jl or JLD2.jl. I believe they both used different modified HDF5 formats.

I just thought that the hdf5.jl package was the best developed, so that is why I chose it. It seems like they use the same syntax, so I would still not know how to save my B array - could you help me out with that?

Kind regards

HDF5 will save arrays of numeric types (eg Int64, Float64, etc) but it won’t save more complicated types by default. You can either save the vector elements yourself like

 B = Vector{Array{Float32,2}}(undef,3)
 B[1] = rand(3,3);
 B[2] = rand(3,3);
 B[3] = rand(3,3);

h5open("B.hdf5", "w") do file
    file["B1"] = B[1]
    file["B2"] = B[2]
    file["B3"] = B[3]
end

Or use a 3d array instead of a vector of 2d arrays like

C = zeros(3,3,3)
C[1,:,:] = rand(3,3);
C[2,:,:] = rand(3,3);
C[3,:,:] = rand(3,3);
h5open("C.hdf5", "w") do file
    file["C"]=C
end

I would recommend HDF5 over JLD if you want your data to be accesible to non-julia users, and if you don’t have super complicated data structures.

2 Likes