# Best way to save/read bytes in struct

**URL:** <https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842>\
**Category:** Performance\
**Tags:** struct\
**Created:** [April 22, 2022, 1:34pm UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842 "2022-04-22T13:34:57Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![nandoconde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nandoconde/32/19497_2.png) [@nandoconde](https://discourse.julialang.org/u/nandoconde)\
**Post date:** [April 22, 2022, 1:34pm UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/1 "2022-04-22T13:34:58Z")

</div>

Hello!

## Main question

I am developing a library to manipulate protocol messages in Julia.

It must be able to store an incoming message, and then return its fields when asked for it. Here is a MWE for it:

```julia
# incoming message: b"\xA7\x05\x61" == 3 bytes
# 
# fields:
# a == byte 0
# b == byte 1, bits 0-3
# c == byte 1, bits 4-7
# d == byte 2

struct msg_A
    payload::Vector{UInt8}
end

function Base.getproperty!(m::msg_A, s::Symbol)
    if s == :a
        return payload[1]
    if s == :b
        return (payload[2] >> 4)
    if s == :c
        return (payload[2] & 0xF)
    if s == :d
        return payload[3]
end

```

Is this the correct approach?

Would it be better storing it as an `IOBuffer`? Or to store it in already separated fields (`a`, `b`, `c` and `d`)?

## Extra question

For some types of messages, such as the one defined above `msg_A`, the length is fixed. Can this fixed length be incorporated to the struct definition (other than defining a `length` function for it)? Will it help the compiler/execution knowing how much it occupies in memory?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [April 22, 2022, 1:38pm UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/2 "2022-04-22T13:38:02Z")

</div>

I’d go with your third option of storing individual fields. It won’t be less efficient but guarantees a fixed size layout & thus helps the compiler generate efficient code. It also preserves type safety.

Julia structs are also compatible with C, layout wise, so you should be able to faithfully represent almost all types.

---

<div class="post-metadata">

**Author:** ![nandoconde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nandoconde/32/19497_2.png) [@nandoconde](https://discourse.julialang.org/u/nandoconde)\
**Post date:** [April 23, 2022, 10:41am UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/3 "2022-04-23T10:41:37Z")

</div>

Thanks Sukera!

And regarding the type for storage, what would be the most efficient/adequate type?

As I see it, short fields I could store in one UInt8 each, or UInt32.

But what if the field I want to store and work with has 40 bits? I guess that could fit in a 64-bit UInt. But what about larger bit fields (70/80, etc)?

In those cases it becomes more difficult working with bits, and bitwise/mask operations (&,|,XOR)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [April 23, 2022, 11:51am UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/4 "2022-04-23T11:51:31Z")

</div>

> [@nandoconde](#):
>
> And regarding the type for storage, what would be the most efficient/adequate type?

That depends on the types in your protocol, but I’d usually go with either directly matching types of equal size or the next-largest type that can fit your elements.

> [@nandoconde](#):
>
> But what about larger bit fields (70/80, etc)?

70/80 bits is a very awkward format to receive and sounds like that’s a composite type (i.e. a type made up of multiple smaller, more primitive types). I’d unpack that from its compressed form to individual fields (or even create its own type for unpacking that into), as unaligned accesses to memory are extremely slow and constantly shifting/masking operations are not really conductive to high performance (this does depend on how much work you do on your received data, but it’s fairly straight forward to implement and easy to use).

---

<div class="post-metadata">

**Author:** ![nandoconde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nandoconde/32/19497_2.png) [@nandoconde](https://discourse.julialang.org/u/nandoconde)\
**Post date:** [April 23, 2022, 11:57am UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/5 "2022-04-23T11:57:39Z")

</div>

That’s a good POV, thanks!

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [April 23, 2022, 1:02pm UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/6 "2022-04-23T13:02:18Z")

</div>

> [@nandoconde](#):
>
> ```julia
> struct msg_A
> payload::Vector{UInt8}
> end
> 
> ```

You could keep the implementation unchanged if you just use a tuple instead of a vector:

```julia
struct msg_A
    payload::NTuple{3, UInt8}
end

```

The other alternatives are ok, except using a `Vector{UInt8}`.

---

<div class="post-metadata">

**Author:** ![krrutkow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krrutkow/32/9138_2.png) [@krrutkow](https://discourse.julialang.org/u/krrutkow)\
**Post date:** [April 23, 2022, 2:51pm UTC](https://discourse.julialang.org/t/best-way-to-save-read-bytes-in-struct/79842/7 "2022-04-23T14:51:52Z")

</div>

If there are already C definitions for the protocol, it is easy to use [CBinding.jl](https://github.com/analytech-solutions/CBinding.jl) to automatically generate the representation in Julia. It produces very efficient code for accessing bitfields and such as well. Here is a [blog post](https://www.analytech-solutions.com/analytech-solutions/blog/cbinding-v1.html) that demonstrates in-place accessing a WAV file just by “including” the format defined in the libsndfile C headers.