How can the Julia language write a structure to a file and read that structure from the file?

General Usage
,
1

How can the Julia language write a structure to a file and read that structure from the file? I tried it myself. Although I could read it correctly, when there are dozens of keywords in the structure, the reading process becomes quite troublesome. Are there any better solutions?

struct MyStruct
    x::Int32
    y::Float64
    z::Array{Float64, 2}
end

s = MyStruct(42, 3.14, reshape(1:16, 4, 4))

fp = "D:\\Radar\\"
cd(fp)

function write_struct(io::IO, s)
    for i in 1:fieldcount(typeof(s))
        write(io, getfield(s, i))
    end
end

open("data.bin", "w") do io
    write_struct(io, s)
end

function read_struct(io::IO, T::Type)
    fields = Vector{Any}(undef, fieldcount(T))
    fields[1] = read(io, fieldtype(T, 1))
    fields[2] = read(io, fieldtype(T, 2))
    M = Matrix{Float64}(undef, 4, 4)
    fields[3] = read!(io, M) # Read binary data from an I/O stream or file, filling in array.

    return T(fields...)
end

open("data.bin", "r") do io
    s_read = read_struct(io, MyStruct)
    println(s_read) 
end
2

I am using:

julia> using Serialization

help?> serialize
...
  serialize(filename::AbstractString, value)

  Open a file and serialize the given value to it.
...

and:

help?> deserialize
...
  deserialize(filename::AbstractString)

  Open a file and deserialize its contents.
...

julia> value = deserialize(filename)

Just works.

3 Likes
3

JSON3.jl has a neat mechanism for doing this. I use it for quite large nested structs and it works great.

2 Likes
4

Have you tried GitHub - JuliaIO/JLD2.jl: HDF5-compatible file format in pure Julia. I have not tried with structs, but have tried with dictionaries and such, and it works great.

4 Likes
5

@curious @TimG @oheil
If I know that a structure has been written in the file, but I don’t know which package performed the serialization, can I use Serializaton, JSON3, or JLD2 to read it? From the documentation, it seems that it is not possible. Serialization and deserialization are one-to-one.

6

If you want to read a file you need to know the file format. Usually this is indicated by the suffix, likejld2 or bin. Sometimes there is also a magic byte at the very beginning of the file that indicates the file format.

the Apache Arrow file format is indicated using a magic byte sequence at both the beginning and the end of the file. Specifically, the file starts and ends with the magic string “ARROW1” (6 bytes), followed by padding bytes to reach an 8-byte boundary. This magic number helps identify files that use the Arrow format and demarcates the file structure, ensuring compatibility and correct parsing.

We use the bin suffix for data stored with serialize. While serialize is fast, it can be problematic because older Julia versions cannot read files written with newer Julia versions.

If you do not know the format you can always try one of the methods. If you get an exception it was the wrong format. If you want to be sure other people can read your file even in 10 or 15 years, use a human readable format like json.

1 Like