# Serializing multiple values

**URL:** <https://discourse.julialang.org/t/serializing-multiple-values/17205>\
**Category:** Internals & Design\
**Tags:** question, serialization\
**Created:** [November 6, 2018, 8:49am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205 "2018-11-06T08:49:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 6, 2018, 8:49am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205/1 "2018-11-06T08:49:18Z")

</div>

The [documentation for Serialization](https://docs.julialang.org/en/stable/stdlib/Serialization/) is pretty sparse, and I am wondering if I am using it correctly. I will write up what I learn here as a PR for the docs.

I want to serialize multiple objects to a stream from an iterator. Should I construct a `Serializer`, then write them individually?

Do I need a header if I keep the object in memory (non-persistent)?

The following works:

```julia
using Serialization

function writedata(itr)
    io = IOBuffer()
    s = Serializer(io)
    Serialization.writeheader(s)
    for elt in itr
        serialize(s, elt)
    end
    take!(io)
end

function readdata(data)
    io = IOBuffer(data)
    while !eof(io)
        @show deserialize(io)
    end
end

```

```julia
julia> data = writedata((42, "a fish", Float64(π)));

julia> readdata(data)
deserialize(io) = 42
deserialize(io) = "a fish"
deserialize(io) = 3.141592653589793

```

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [November 6, 2018, 10:48am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205/2 "2018-11-06T10:48:23Z")

</div>

I usually serialize/deserialize entire tuples containing all objects I want to send or store. This requires no extra machinery. I guess you could `collect` your iterator and do the same?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 6, 2018, 10:55am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205/3 "2018-11-06T10:55:23Z")

</div>

No, I specifically don’t want to collect. I am dealing with a large amount of data, obtained online and with an unknown eltype, and I found I can compress the serialized stream using

> **[GitHub - JuliaIO/CodecZlib.jl: zlib codecs for TranscodingStreams.jl.](https://github.com/JuliaIO/CodecZlib.jl)**
>
> zlib codecs for TranscodingStreams.jl. Contribute to JuliaIO/CodecZlib.jl development by creating an account on GitHub.

very efficiently.

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [March 13, 2019, 12:29am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205/4 "2019-03-13T00:29:04Z")

</div>

Should this also work for serializing custom structs?

I tried with this simple example but get an error when trying to `deserialize`.

Code as above with this:

```julia
struct Foo
    a::Float64
end

itr = [Foo(i) for i = 1:3]
data = writedata(itr)
readdata(data)

```

Seems to successfully `deserialize` the first element but errors on the second.

```julia
Julia-1.1.0> readdata(data)
deserialize(io) = Foo(1.0)
ERROR: KeyError: key 0 not found
Stacktrace:
 [1] getindex at .\abstractdict.jl:599 [inlined]
 [2] handle_deserialize(::Serializer{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Int32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Serialization\src\Serialization.jl:764
 [3] deserialize(::Serializer{Base.GenericIOBuffer{Array{UInt8,1}}}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Serialization\src\Serialization.jl:731
 [4] handle_deserialize(::Serializer{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Int32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Serialization\src\Serialization.jl:778
 [5] readdata(::Array{UInt8,1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Serialization\src\Serialization.jl:731
 [6] top-level scope at none:0

```

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [March 16, 2019, 8:39am UTC](https://discourse.julialang.org/t/serializing-multiple-values/17205/5 "2019-03-16T08:39:07Z")

</div>

The data was written with the same `Serializer` object, so there can be shared references within the stream. So the data should be deserialized with the same `Serializer` too.

Note that `deserialize(io::IO)` creates a new `Serializer` object each time, which causes the error above.

`readdata` should be:

```julia
function readdata(data)
    io = IOBuffer(data)
    s = Serializer(io)
    while !eof(io)
        @show deserialize(s)
    end
end

```

Ref: [Error deserializing multiple values of custom struct · Issue #31337 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/31337)