# Allocations when using getfield with a tuple/vector of symbols

**URL:** <https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756>\
**Category:** Performance\
**Tags:** memory-allocation, struct\
**Created:** [November 18, 2024, 4:26am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756 "2024-11-18T04:26:34Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 18, 2024, 4:26am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/1 "2024-11-18T04:26:34Z")

</div>

MWE:

```julia
using Parameters, BenchmarkTools
@with_kw struct Test
    A :: String = "A"
    B :: Int64 = 0
end

test = Test()
tup = (:A, :B)

@btime getfield($test, :A)
@btime getfield($test, $tup[1])

```

Is there a way to loop over struct fields without these allocations?

```julia
julia> @btime getfield($test, :A)
  2.400 ns (0 allocations: 0 bytes)
"A"

julia> @btime getfield($test, $tup[1])
  23.002 ns (1 allocation: 32 bytes)
"A"

```

---

<div class="post-metadata">

**Author:** ![DatName](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datname/32/418_2.png) [@DatName](https://discourse.julialang.org/u/DatName)\
**Post date:** [November 18, 2024, 7:20am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/2 "2024-11-18T07:20:17Z")

</div>

You could use generated function:

```julia
@generated function getfield_unrolled(t::T, f::Symbol) where {T}
    names = fieldnames(T)
    exprs = [:($(QuoteNode(name)) == f && return getfield(t, $(QuoteNode(name)))) for (c, name) in enumerate(names)]

    push!(exprs, :(throw(ErrorException("type $T has no field $f"))))
    return quote
        $(exprs...)
    end
end

```

and get

```julia
julia> @btime getfield_unrolled($test, $(tup[1]))
  1.855 ns (0 allocations: 0 bytes)
"A"

```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 18, 2024, 10:14am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/3 "2024-11-18T10:14:49Z")

</div>

And finally I know a good example showing that the allocation can be for the ~~runtime dispatch~~ alone, not the uncertain return type. Still no idea what that allocation is doing though. EDIT: actually `getfield` is a builtin function so I’m not even sure if it runtime dispatches the way generic functions do.

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 18, 2024, 11:53am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/4 "2024-11-18T11:53:54Z")

</div>

Thanks, I’ve been trying to avoid metaprogramming for this project and manually unrolling such loops, but it can be repetitive/tedious. Maybe the warnings against metaprogramming are overblown?

It seems like a lot of trouble for a seemingly simple task.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [November 18, 2024, 12:06pm UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/5 "2024-11-18T12:06:36Z")

</div>

> [@Tetrakai](#):
>
> It seems like a lot of trouble for a seemingly simple task.

No. Metaprogramming is appropriate for this task.

There is fundamentally no way of making field access fast if the field name is not known at compile time. In other words, you must make sure that the field name is known at compile time. This is fundamentally metaprogramming.

There are three approaches:

1. Constant propagation: `getfield(x, :A)` and similar.
2. Lifting to type domain: `myGetfield(x, ::Val{sym}) where sym = getfield(x, sym)`
3. explicit metaprogramming

Lifting to type domain is a way of explicitly encouraging constant propagation at various points.

The issue with naively relying on const-prop is that you are at the mercy of unstable compiler heuristics. It forces all readers of your code to understand how const-prop and inlining works in all julia versions you support and figure out in their head whether that applies to your construction. This is terrible for maintainability. Better go for metaprogramming if it is not exceedingly obvious that the field-names are known at compile time.

For that, you must benchmark differently: Don’t benchmark “the small function you want to measure”. For tiny functions, it’s all about interaction with context (surrounding code), so you must write a realistic outer loop function that calls your tiny function, and benchmark that. Benchmarking / performance is _not composable_ for small timings.

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 18, 2024, 2:28pm UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/6 "2024-11-18T14:28:48Z")

</div>

For this case the field names are known ahead of time. I know exactly which elements will be extracted.

Ie, this works fine:

```julia
a = test.A
b = test.B

```

But doing this for, eg, 10 fields becomes repetitive.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [November 18, 2024, 2:39pm UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/7 "2024-11-18T14:39:23Z")

</div>

> [@Tetrakai](#):
>
> But doing this for, eg, 10 fields becomes repetitive.

So why not implement tuple unpacking such that you can write `a,b = Test`? For that you need to extend `Base.indexed_iterate` for your type, either by hand or by metaprogramming a la `@with_kw`.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [November 19, 2024, 7:28am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/8 "2024-11-19T07:28:41Z")

</div>

Couldn’t you just do

```julia
(;a, b) = my_object

```

without implementing any additional methods?

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 19, 2024, 8:16am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/9 "2024-11-19T08:16:16Z")

</div>

If I understand correctly, those suggestions help with the MWE but the real code would be just as verbose as if I manually unroll it.

Eg, I need to add something (from another struct I am looping the same way) to `a` and store it, then `b`, etc. My purpose is to avoid repetitive code without sacrificing performance.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [November 19, 2024, 1:21pm UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/10 "2024-11-19T13:21:41Z")

</div>

Generally, for looping over fields/properties, one would extract them as a namedtuple – then, stuff like `map` works and works performantly.  
For extraction, use ConstructionBase.jl: it has `getfields(x)::NamedTuple` and `getproperties(x)::NamedTuple`.

It may also help if you show some specific examples of what you are trying to achieve.

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 19, 2024, 8:58pm UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/11 "2024-11-19T20:58:49Z")

</div>

Sure, here is one of the structs in question:

```julia
@with_kw struct Roster{T1 <: SVector{MAXPLAYERS, String15}, 
                       T2 <: SVector{MAXPLAYERS, Int16}}
    # Player Info
    Name :: T1 = @SVector fill(String15(""), MAXPLAYERS)
    Age :: T1 = @SVector fill(String15(""), MAXPLAYERS)
    Nat :: T1 = @SVector fill(String15(""), MAXPLAYERS)
    Prs :: T1 = @SVector fill(String15(""), MAXPLAYERS)

    # Ratings
    St :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Tk :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Ps :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Sh :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Sm :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Ag :: T2 = @SVector fill(Int16(0), MAXPLAYERS)

    # Abilities
    KAb :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    TAb :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    PAb :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    SAb :: T2 = @SVector fill(Int16(0), MAXPLAYERS)

    # Stats
    Gam :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Sav :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Ktk :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Kps :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Sht :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Gls :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Ass :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    DP :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Inj :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Sus :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
    Fit :: T2 = @SVector fill(Int16(0), MAXPLAYERS)
end

```

And one of the functions (using the solution recommended above):

```julia
function calc_metric(baseline, sims)
    nteams = length(baseline.lg)
    nreps = length(sims)
    pl_fields = (:Gam, :Sav, :Ktk, :Kps, :Sht, :Gls, :Ass, :DP)
    tm_fields = (:Pl, :W, :D, :L, :GF, :GA, :GD, :Pts)

    sumSq = 0
    for sim in sims
        # Player Stats
        for i in eachindex(pl_fields)
            for j in eachindex(baseline.lg)
                x = getfield_unroll(baseline.lg[j].roster, pl_fields[i])
                y = getfield_unroll(sim.lg[j].roster, pl_fields[i])
                sumSq += sum((Int64.(x - y)).^2)
            end
        end

        # Team Stats
        for i in eachindex(tm_fields)
            for j in eachindex(baseline.lg_table)
                x = getfield_unroll(baseline.lg_table[j], tm_fields[i])
                y = getfield_unroll(sim.lg_table[j], tm_fields[i])
                sumSq += sum((Int64.(x - y))^2)
            end
        end

    end

    # Refers to RMSE per team (not total number of variables)
    RMSE = sqrt(sumSq/(nteams*nreps))

    return RMSE
end

```

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [November 20, 2024, 2:12am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/12 "2024-11-20T02:12:03Z")

</div>

> [@Tetrakai](#):
>
> ```julia
> # Player Stats
> for i in eachindex(pl_fields)
> for j in eachindex(baseline.lg)
> x = getfield_unroll(baseline.lg[j].roster, pl_fields[i])
> y = getfield_unroll(sim.lg[j].roster, pl_fields[i])
> sumSq += sum((Int64.(x - y)).^2)
> end
> end
> 
> ```

Perhaps old-fashioned broadcasting will get similar results:  
(couldn’t actually test this, there may be tweaking needed because values are Arrays)

```julia
using LinearAlgebra # to use `dot`

# Player stats
# broadcasting over pl_fields, so no second `for`
for j in eachindex(baseline.lg)
    x = Int64.(getfield.(Ref(baseline.lg[j].roster), pl_fields))
    y = Int64.(getfield.(Ref(sim.lg[j].roster, pl_fields))
    sumSq += sum(t -> dot(t,t), x .- y)
end

```

(the other loop can be modified in a similar way)

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [November 20, 2024, 2:45am UTC](https://discourse.julialang.org/t/allocations-when-using-getfield-with-a-tuple-vector-of-symbols/122756/13 "2024-11-20T02:45:24Z")

</div>

I feel something like

```julia
pl_type = NamedTuple{(:Gam, :Sav, :Ktk, :Kps, :Sht, :Gls, :Ass, :DP)}
...
p1 = getproperties(baseline.lg[j].roster)
p2 = getproperties(sim.lg[j].roster)
sumSq += map(pl_type(p1), pl_type(p2)) do x, y
    abs2(x - y)
end |> sum
...

```

should be quite performant, and more readable.  
Would be easier to check with an MWE of course 🙂

Also, you may want to consider using a StructArray instead of your `Roster` struct. Or, if you do want to dispatch on `::Roster`, to have this struct only with a `data::StructArray` field.