# Reinterpret as a complex struct

**URL:** <https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784>\
**Category:** General Usage\
**Created:** [May 30, 2019, 8:22pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784 "2019-05-30T20:22:10Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [May 30, 2019, 8:22pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/1 "2019-05-30T20:22:10Z")

</div>

Hi! There is a dataset loaded by a 3rd-party library in the form of a raw float array. I want to view this data as a vector of structs `A`, each having layout like this: `<F32><F32><F32>(B*8*60*3*4)<F32>`, i.e. three floats, then a `8*60*3*4` array of structs `B` and then another float. Struct `B` is simply three floats.  
However I cannot find a reasonable way to do this using `reinterpret` or otherwise. `reinterpret` requires `bitstype`, so something like `StaticArrays` for the inner arrays. However, `StaticArrays` has poor performance, especially compilation part, for arrays that large.  
Any other way to view a raw buffer array as an array of large nested structs?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 31, 2019, 5:54am UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/2 "2019-05-31T05:54:43Z")

</div>

I am not sure I understand the specs, but what about

```julia
data = Float64.(1:(3 + 3*8*60*3*4 + 1))

struct B{T}
    a::T
    b::T
    c::T
end

header = data[1:3]
Bs = reinterpret(B{Float64}, @view data[4:(end - 1)])
footer = data[end]

```

That said, a static array type instead of `B{Float64}` should be fine too.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [May 31, 2019, 1:46pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/3 "2019-05-31T13:46:42Z")

</div>

I mean something more along these lines:

```julia
struct B
  a::Float32
  b::Float32
  c::Float32
end

struct A
  a::Float32
  b::Float32
  c::Float32
  d::StaticArray{Tuple{3,8,60,3,4}, B}
  e::Float32
end

reinterpret(A, data)

```

This basically works, but many operations are slow due to the large staticarray.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 31, 2019, 1:58pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/4 "2019-05-31T13:58:15Z")

</div>

I don’t understand the motivation to use static arrays like this, this is known to be suboptimal.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [May 31, 2019, 2:00pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/5 "2019-05-31T14:00:31Z")

</div>

Sure, this was just to explain what I need. Replacing StaticArray with Array does not work here, because `reinterpret` requires bits type.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 31, 2019, 2:01pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/6 "2019-05-31T14:01:44Z")

</div>

Is there a reason you can’t do what I suggested above?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [May 31, 2019, 2:04pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/7 "2019-05-31T14:04:33Z")

</div>

I would really like to use the resulting array `x` as `x[i].a`, `x[i].d[a, b, c, d, e]` and so on. As I understand, your code handles a single element of this (single `A` struct), and cannot be applied to the whole array of them.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [May 31, 2019, 9:35pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/8 "2019-05-31T21:35:02Z")

</div>

I don’t think what you’re asking for is possible without gigantic static arrays (or an equivalent giant bitstype) because the memory layout of the natural non-bitstype julia structs would simply be different than the layout of your vector.

But there may still be a way to get what you want. If your goal is just to avoid expensive copying, then perhaps you can construct each `d` field from a `reinterpret(B, view(data, i, j))` where i and j are the relevant indices for each `A`’s data. You’d still need to copy the extra floats in each A, but that should be very cheap.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [June 1, 2019, 11:23am UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/9 "2019-06-01T11:23:32Z")

</div>

Thanks! This is probably the closest possible solution. However, copying the floats has an effect that when they are modified by assigning to a field in the resulting array (of `mutable struct`s), these changes do not propagate to the original data buffer. Currently this is good enough for me, but I’m still interested if someone can think of a way to make a true view of a raw buffer array as an array of complex structs like in this example.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [June 1, 2019, 1:58pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/10 "2019-06-01T13:58:47Z")

</div>

The most realistic variant is something like

```julia
julia> struct arrPtr<:AbstractArray{Float32, 4}
       ptr::Ptr{Float32}
       end

julia> Base.size(::arrPtr) = (8, 60, 3, 4)
julia> Base.getindex(a::arrPtr, i) = unsafe_load(a.ptr, i)
julia> Base.setindex!(a::arrPtr, x, i) = unsafe_store!(a.ptr, x, i)
julia> Base.IndexStyle(::Type{arrPtr}) = Base.IndexLinear()

julia> struct Bptr
       ptr::Ptr{Float32}
       end

julia> function Base.getproperty(b::Bptr, s::Symbol)
       ptr = getfield(b, 1)
       if s == :a 
         return unsafe_load(ptr, 1)
       elseif s==:b
         return unsafe_load(ptr, 2)
       elseif s==:c
         return unsafe_load(ptr, 3)
       elseif s==:d
         return arrPtr(ptr + 12)
       elseif s==:e
         return unsafe_load(ptr, 3 + 5760)
       else
         error()
       end
       end

julia> function Base.setproperty!(b::Bptr, s::Symbol, v::Float32)
       ptr = getfield(b, 1)
       if s == :a 
         return unsafe_store!(ptr, v, 1)
       elseif s==:b
         return unsafe_store!(ptr, v, 2)
       elseif s==:c
         return unsafe_store!(ptr, v, 3)
       elseif s==:e
         return unsafe_store!(ptr, v, 3 + 5760)
       else
         error()
       end
       end

julia> struct asBptr<:AbstractVector{Bptr}
       ptr::Ptr{Float32}
       len::Int
       keepalive::Any
       end
julia> asBptr(arr::Array{Float32}) = asBptr(pointer(arr), div(length(arr), 16+4*5760), arr)
julia> Base.size(buf::asBptr) = (buf.len,)
julia> Base.getindex(buf::asBptr, i) = Bptr(buf.ptr + (16 + 4*5760)*(i-1))
julia> Base.IndexStyle(::Type{asBptr}) = Base.IndexLinear()

```

This gives use like

```julia
julia> ab = asBptr(arr)
43-element asBptr:
 Bptr(Ptr{Float32} @0x00007f3acb016040)
 Bptr(Ptr{Float32} @0x00007f3acb01ba50)
...
julia> ab[14].a
0.4085027f0
julia> ab[14].d[:, 2, 2, 2]
8-element Array{Float32,1}:
 0.15881789
 0.41884196
 0.07280159
 0.8976873 
 0.97513235
 0.2145493 
 0.5796931 
 0.08459842

julia> ab[14].d[:, 2, 2, 2].=0;

```

This kind of approach is effectively without alternative if you deal with a memory mapped file and mutations need to be visible to other processes.

Note that there are no boundschecks. `asBptr` keeps the underlying storage alive, but `Bptr` or `arrPtr` don’t (so make sure that the garbage collector does not steal your storage away!).

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [June 14, 2019, 3:37pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/11 "2019-06-14T15:37:19Z")

</div>

However, this is not type stable, right? Due to the `if` block in `getproperty` returning different types that depend on value of `s` passed.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [June 14, 2019, 3:41pm UTC](https://discourse.julialang.org/t/reinterpret-as-a-complex-struct/24784/12 "2019-06-14T15:41:50Z")

</div>

If a function like that `getproperty` method above is called with a _constant_ (as is the case when you do `foo.bar`), the compiler can propagate that constant through the function and figure out the return type, even in cases that look type-unstable. For example:

```julia
julia> function looks_type_unstable(s::Symbol)
         if s == :a
           1.0
         else
           "hello"
         end
       end
looks_type_unstable (generic function with 1 method)

julia> function passes_a_constant()
         looks_type_unstable(:a)
       end
passes_a_constant (generic function with 1 method)

julia> @code_warntype(passes_a_constant())
Body::Float64
1 ─ return 1.0

```