# Filling a StructArray in a type-stable manner

**URL:** <https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319>\
**Category:** Performance\
**Tags:** structarrays\
**Created:** [March 7, 2024, 7:44pm UTC](https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319 "2024-03-07T19:44:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)\
**Post date:** [March 7, 2024, 7:44pm UTC](https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319/1 "2024-03-07T19:44:18Z")

</div>

I am trying to create `StructArray`’s reading from a ROOT file with the `UnROOT.jl` package, which provides a `NamedTuple` interface, in a type-stable manner. The following code illustrates the problem:

```julia
using StructArrays
using BenchmarkTools

struct Vector3f
    x::Float32
    y::Float32
    z::Float32
end

struct Foo
    a::Int64
    b::Int64
    c::Int64 # not in data
    vector::Vector3f
end

function StructArray{T,M}(dat::NamedTuple) where {T <: Number, M} # number types
    getproperty(dat, M)
end
function StructArray{T,M}(dat::NamedTuple) where {T,M} # generic composite types
    len = length(dat[1])
    tup = Tuple( map(zip(fieldnames(T),fieldtypes(T))) do (fn,ft)
                    if fn == :c
                        fill(0,len) # missing in data
                    else
                        StructArray{ft, Symbol(M, :_ ,fn)}(dat)
                    end
                end
        )
    StructArray{T}(tup)
end

function StructArray{Foo,M}(dat::NamedTuple, ::Bool) where {M} # very concrete for Foo type
    tup = (
        getproperty(dat, Symbol(M,:_a)),
        getproperty(dat, Symbol(M,:_b)),
        fill(0,100),
        StructArray{Vector3f}(( getproperty(dat, Symbol(M,:_vector_x)), 
                                getproperty(dat, Symbol(M,:_vector_y)),
                                getproperty(dat, Symbol(M,:_vector_z)) ))
    )
    StructArray{Foo}(tup)
end

data = (Foos_a=fill(1,100), 
        Foos_b=fill(2,100), 
        Foos_vector_x=fill(1.,100),
        Foos_vector_y=fill(2.,100),
        Foos_vector_z=fill(3.,100))

```

I get

```julia
julia> @btime foos1 = StructArray{Foo, :Foos}(data);
  10.000 μs (101 allocations: 5.67 KiB)

julia> @btime foos2 = StructArray{Foo, :Foos}(data, true);
  152.556 ns (2 allocations: 960 bytes)

```

The first uses a generic implementation, which is what I would want for my application, but it is factors slower that the second implementation, which is written explicitly for the concrete type `Foo`.  
Does anybody has an idea how I would write `StructArray{T,M}(dat::NamedTuple) where {T,M}` in a type-stable manner? Thanks.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [March 7, 2024, 7:46pm UTC](https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319/2 "2024-03-07T19:46:44Z")

</div>

is that really a bottleneck? I think ideally we should do type-space computation “once” near the beginning of the whole workload, and then never worry about it.

In other words, as long as this doesn’t scale with the amount of data we process, it’s not an issue I think

---

<div class="post-metadata">

**Author:** ![peremato](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/peremato/32/29128_2.png) [@peremato](https://discourse.julialang.org/u/peremato)\
**Post date:** [March 7, 2024, 7:53pm UTC](https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319/3 "2024-03-07T19:53:03Z")

</div>

At least this is a reproducible bottleneck. I need to do each event the operation of combining a set of vectors into a `StructArray`. I do not know how to do something at the beginning and reuse it over an over later for each event.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [March 7, 2024, 8:05pm UTC](https://discourse.julialang.org/t/filling-a-structarray-in-a-type-stable-manner/111319/4 "2024-03-07T20:05:11Z")

</div>

> [@peremato](#):
>
> ```julia
> tup = Tuple( map(zip(fieldnames(T),fieldtypes(T))) do (fn,ft)
> if fn == :c
> fill(0,len) # missing in data
> else
> StructArray{ft, Symbol(M, :_ ,fn)}(dat)
> end
> end
> )
> 
> ```

I just don’t see how this can avoid allocation.

I think if you work with RNTuple, we can come up a way to expose the storage “column” directly, so you don’t have to work with NamedTuples maybe. As it is right now, only way I can think of is if you do `@generated`