# Unpacking binary data into a Julia struct

**URL:** <https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941>\
**Category:** General Usage\
**Tags:** question, binaryio\
**Created:** [January 3, 2020, 10:00pm UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941 "2020-01-03T22:00:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![tlnagy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlnagy/32/5815_2.png) [@tlnagy](https://discourse.julialang.org/u/tlnagy)\
**Post date:** [January 3, 2020, 10:00pm UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/1 "2020-01-03T22:00:59Z")

</div>

I’m having a hard time finding an example in Julia v1.0+ for handling unpacking binary data into a Julia struct. Is there an easy way to do this?

Say I have some custom struct `Test` and wanted to read some binary data from stream `io`, is there someway to fill all the fields of a test struct very quickly?

```julia
julia> struct Test
           a::UInt16
           b::UInt16
           c::UInt32
       end

julia> read(io, Test)
ERROR: The IO stream does not support reading objects of type Test.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] read(::IOStream, ::Type) at ./io.jl:917
 [3] top-level scope at REPL[40]:1

julia> Test([read(io, t) for t in fieldtypes(Test)]...) #something that isn't this....
Test(0x002a, 0x0008, 0x2a270010)

```

---

<div class="post-metadata">

**Author:** ![Daniel\_Berge](https://avatars.discourse-cdn.com/v4/letter/d/eb9ed0/32.png) [@Daniel\_Berge](https://discourse.julialang.org/u/Daniel_Berge)\
**Post date:** [January 3, 2020, 10:14pm UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/2 "2020-01-03T22:14:27Z")

</div>

You need to define how to read the data from the stream.

```julia
function Base.read(io::IO, ::Type{Test})
    Test(read(io,UInt16),read(io,UInt16),read(io,UInt32))
end

```

As an example, this is how `Complex` numbers are handled in `Base`.

```julia
function read(s::IO, ::Type{Complex{T}}) where T<:Real
    r = read(s,T)
    i = read(s,T)
    Complex{T}(r,i)
end

```

---

<div class="post-metadata">

**Author:** ![tlnagy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlnagy/32/5815_2.png) [@tlnagy](https://discourse.julialang.org/u/tlnagy)\
**Post date:** [January 3, 2020, 11:06pm UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/3 "2020-01-03T23:06:38Z")

</div>

That’s a pretty elegant solution that slipped my mind. I guess I thought there was a way without implementing my own function for reading

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [January 4, 2020, 12:16am UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/4 "2020-01-04T00:16:25Z")

</div>

I would suggest to consider the native `Serialization` solution. See

```julia
using Serialization: serialize, deserialize

struct Test
   a::UInt16
   b::UInt16
   c::UInt32
end

test = Test(0, 0, 0)

# write to io
open("plsdel.io", "w") do io
   serialize(io, test)
end

# read from io
test_deserialized = open("plsdel.io", "r") do io
   deserialize(io)
end

test == test_deserialized # true

```

This is less manual and likely to be well-support as `Serialization` is in base.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [January 4, 2020, 1:31am UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/5 "2020-01-04T01:31:19Z")

</div>

Super clever solution. I’ve been reading in binary data in ad-hocish ways.

One approach:  
Iterate over the field names and types in the struct. Yes this works, I just forget the function to get field names and types from structs right now 😃 . So defining the struct, defines the types, and its all stored sequentially.

It falls apart a bit with Chars/strings, buuuut you can patch that with a dict if you know the length.

---

<div class="post-metadata">

**Author:** ![miromarszal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/miromarszal/32/6721_2.png) [@miromarszal](https://discourse.julialang.org/u/miromarszal)\
**Post date:** [July 14, 2020, 6:43pm UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/6 "2020-07-14T18:43:29Z")

</div>

Interestingly, loading into an `Array` of `Test` with `read!` will work:

```julia
A = Array{Test}(undef, 1)
read!(io, A)

```

Somehow `read!` knows how to read from the stream, but `read` doesn’t. Is it intended to be this way?

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [November 25, 2020, 1:22am UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/7 "2020-11-25T01:22:43Z")

</div>

Bumping this question–can I rely on `read!(f, Vector{MyStruct}(undef, 1))[1]`, or might there be unexpected sharp edges? The documentation is pretty terse:

```julia
help?> read!

  read!(stream::IO, array::AbstractArray)
  read!(filename::AbstractString, array::AbstractArray)

  Read binary data from an I/O stream or file, filling in array.

```

…alignment? padding?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 25, 2020, 8:43am UTC](https://discourse.julialang.org/t/unpacking-binary-data-into-a-julia-struct/32941/8 "2020-11-25T08:43:17Z")

</div>

I think that [you get C-style padding and alignment](https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/#Struct-Type-Correspondences). Cf

[https://github.com/pao/StrPack.jl](https://github.com/pao/StrPack.jl)

and similar packages for alternantives.