Serializing multiple values

question

#1

The documentation for Serialization is pretty sparse, and I am wondering if I am using it correctly. I will write up what I learn here as a PR for the docs.

I want to serialize multiple objects to a stream from an iterator. Should I construct a Serializer, then write them individually?

Do I need a header if I keep the object in memory (non-persistent)?

The following works:

using Serialization

function writedata(itr)
    io = IOBuffer()
    s = Serializer(io)
    Serialization.writeheader(s)
    for elt in itr
        serialize(s, elt)
    end
    take!(io)
end

function readdata(data)
    io = IOBuffer(data)
    while !eof(io)
        @show deserialize(io)
    end
end
julia> data = writedata((42, "a fish", Float64(π)));

julia> readdata(data)
deserialize(io) = 42
deserialize(io) = "a fish"
deserialize(io) = 3.141592653589793

#2

I usually serialize/deserialize entire tuples containing all objects I want to send or store. This requires no extra machinery. I guess you could collect your iterator and do the same?


#3

No, I specifically don’t want to collect. I am dealing with a large amount of data, obtained online and with an unknown eltype, and I found I can compress the serialized stream using


very efficiently.