# Efficient reflection on structs

**URL:** <https://discourse.julialang.org/t/efficient-reflection-on-structs/55278>\
**Category:** Performance\
**Created:** [February 14, 2021, 9:24pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278 "2021-02-14T21:24:53Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 9:24pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/1 "2021-02-14T21:24:53Z")

</div>

Hello,

I am trying to do something to StructArray but I can’t seem to generate efficient code.

Let’s say there is `struct Wrapper{T}` and it is supposed to behave similarly to `T` except I’m managing where/how the underlying fields are stored.

I have a working version that use `getproperty`, `fieldnames` and `fieldtype`. The problem is that the compiler is unable to do any optimization and performance is really poor.

My second idea was to pass all the information needed as part of the type parameters (NamedTuple with types and some extra info I need) and compute this once and for all before instantiating the wrapper. The problem is that I can’t pass any structure that has a `DataType` as part of a type parameter.

How to generate efficient code for this kind of application ?

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 14, 2021, 9:26pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/2 "2021-02-14T21:26:58Z")

</div>

Generally speaking, the compiler is very good at dealing with this sort of thing. Could you provide a minimal working example of the problem you’re seeing? It’s hard to advise on this sort of stuff in abstract terms.

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 9:27pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/3 "2021-02-14T21:27:38Z")

</div>

Of course! Give me a minute and I’ll give a simplified example

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 9:37pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/4 "2021-02-14T21:37:08Z")

</div>

```julia
module Storage

struct Wrapper{T}
    d::T
end

function Base.getproperty(wrapper::Wrapper{T}, s::Symbol) where {T}
    fields = fieldnames(T)
    if s in fields
        3
    end
    2
end

end

struct Data
    var_1::Int
    var_2::Float32
end

a = Data(1, 0.5)
b = Storage.Wrapper{Data}(a)

```

and the timings:

```julia
julia> @btime a.var_1
  31.502 ns (0 allocations: 0 bytes)
1

julia> @btime b.var_1
  170.836 ns (1 allocation: 32 bytes)
2

```

To be honest I’m even surprised there is a single allocation there. It’s supposed to return a constant after all optimizations are applied. It should be faster than accessing the struct.

Looking at the code llvm I’m also wondering why is there still a call to fieldnames. Given a type {T} it’s constant. Why is there no constant propagation ?

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 14, 2021, 9:46pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/5 "2021-02-14T21:46:10Z")

</div>

I think the issue is that constant propagation doesn’t work through the `in` function, its just too complex. For your MWE, this fixes it though:

```julia
function Base.getproperty(wrapper::Wrapper{T}, s::Symbol) where {T}
    if hasfield(T,s)
        3
    end
    2
end

```

Note also that to enable constant propagation you need the thing you’re benchmarking inside a function, so something like:

```julia
@btime (a -> a.var_1)(a)
@btime (b -> b.var_1)(b)

```

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 9:52pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/6 "2021-02-14T21:52:17Z")

</div>

Oh that’s smart! Thank you.

What is “too complicated”?. Is there a way to determine which ones to track down without taking time from people on the forums?

Also very weird:

```julia
function Base.getproperty(wrapper::Wrapper{T}, s::Symbol) where {T}
    fields = fieldnames(T)
    if hasfield(T, s)
        3
    end
    2
end

```

Is still slow. I left `fields` by mistake but I thought Julia is doing dead code elimination. What is going on there?

Thanks!

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 14, 2021, 9:59pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/7 "2021-02-14T21:59:54Z")

</div>

Sorry, I think I gave the wrong reason, the reason your original code was slow was simply because it was type unstable, because of `fieldnames`,

```julia
@code_warntype b.var_1

Variables
  #self#::Core.Const(getproperty)
  wrapper::Wrapper{Data}
  s::Symbol
  fields::Tuple{Vararg{Symbol, N} where N}

Body::Int64
1 ─ (fields = Main.fieldnames($(Expr(:static_parameter, 1))))
│ %2 = (s in fields)::Bool
└── goto #2 if not %2
2 ─ return 2

```

I guess I’m slightly surprised by that, that seems like it could be fixed. Your dead code elimination is probably because `fieldnames` isn’t pure so the compiler doesn’t know that it doesn’t have other side-effects.

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 10:05pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/8 "2021-02-14T22:05:03Z")

</div>

Why was it type-unstable ? `fieldnames` is just a Tuple of Symbols and the length of the tuple is constant given T.

Why isn’t it pure either ? I thought by convention all functions without `!` were pure. Since it’s a built-in I would assume it should follow the convention?

Thanks again. This is really helping me understand the language.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 14, 2021, 10:20pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/9 "2021-02-14T22:20:31Z")

</div>

> [@Guillaume\_Leclerc](#):
>
> Why was it type-unstable ? `fieldnames` is just a Tuple of Symbols and the length of the tuple is constant given T.

Its just an issue with Julia, not your usage of it, it just looks like `fieldnames` is not written in a way thats type stable (you’ll note above the length of the tuple is not inferred, since `N` is a free variable). It definitely seems like this could be improved, my guess as to why it is this way is just that `fieldnames` wasn’t meant to be used in performance-critical code, instead you have `hasfield` for cases like yours.

> [@Guillaume\_Leclerc](#):
>
> Why isn’t it pure either ? I thought by convention all functions without `!` were pure. Since it’s a built-in I would assume it should follow the convention?

The `!` is just a loose convention, there’s a separate much more strict definition of [@pure](https://docs.julialang.org/en/v1/base/base/#Base.@pure). (I should probably mention that I’m not an expert on this, I think there’s probably cases where LLVM can do dead-code elimination on stuff inside non-pure Julia functions if it can figure out there’s no side-effects, that just didn’t happen in your example).

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 10:29pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/10 "2021-02-14T22:29:42Z")

</div>

Thanks! So the takeaway is essentially that it should be fast in theory but it is not.

I will of course use `hasfield` but what is the thing to do in that case? Do people file issues or just go around the problem when they encounter one ?

---

<div class="post-metadata">

**Author:** ![FPGro](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fpgro/32/20822_2.png) [@FPGro](https://discourse.julialang.org/u/FPGro)\
**Post date:** [February 14, 2021, 11:13pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/11 "2021-02-14T23:13:44Z")

</div>

Search the GitHub issues if this has been discussed before, maybe there is a non-trivial reason for this to be the way it is. But most likely this was just not a priority, so by all means: do file an issue if there is none already!

Oh and the `!` is not about purity, it’s about mutating arguments. Take `print`, not mutating in the common sense but not pure either.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 14, 2021, 11:14pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/12 "2021-02-14T23:14:22Z")

</div>

Agree, I think it couldn’t hurt filing an Issue, although I’m not sure it’d be super high priority. There does look to be some related discussion in

- [Should fieldnames return a tuple rather than an array? · Issue #25327 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/25327)
- [Change fieldnames() and propertynames() to return a tuple rather than an array by nalimilan · Pull Request #25725 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/25725)

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 14, 2021, 11:15pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/13 "2021-02-14T23:15:38Z")

</div>

> [@FPGro](#):
>
> Oh and the `!` is not about purity, it’s about mutating arguments. Take `print` , not mutating in the common sense but not pure either.

Good point. I don’t know why I didn’t think of that…

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [February 14, 2021, 11:24pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/14 "2021-02-14T23:24:27Z")

</div>

> [@FPGro](#):
>
> Oh and the `!` is not about purity, it’s about mutating arguments. Take `print` , not mutating in the common sense but not pure either.

I think that is more that anything that deals with IO is assumed to be mutating and the founders found to be a bother to suffix every IO function.

---

<div class="post-metadata">

**Author:** ![FPGro](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fpgro/32/20822_2.png) [@FPGro](https://discourse.julialang.org/u/FPGro)\
**Post date:** [February 15, 2021, 12:11am UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/15 "2021-02-15T00:11:28Z")

</div>

Well, IO manipulation is commonly considered a side-effect, but I see your point.  
Mine still stands, `!` indicated that one argument will be directly mutated.  
Purity is a different rabbit hole, there are various shades of purity and most of the commonly found functions are not pure in the strict sense.

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 15, 2021, 1:24am UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/16 "2021-02-15T01:24:48Z")

</div>

I don’t know if I should make another post or continue here but here I go.

Any idea why this is not type-stable either:

```julia
struct Test
    a::Int
    b::Float32
end

function init(::Type{T}, s::Symbol) where T
    tpe = fieldtype(T, s)
    zero(tpe)
end

@inferred init(Test, :a)

```

- `init` is parametric on `T`
- `s` is a symbol so it should be able to dispatch on it
- for a given `T` and `s`, `tpe` is constant

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 15, 2021, 2:02am UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/17 "2021-02-15T02:02:35Z")

</div>

Same reason as above, the code needs to be in a function for Julia to try and propagate the constant value `:a` (which it needs to do to infer this). You can combine putting it in a function and running `@inferred` in one line like:

```julia
@inferred (() -> init(Test, :a))()

```

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 15, 2021, 4:09pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/18 "2021-02-15T16:09:20Z")

</div>

Thanks again @marius311 . The `Base.pure()` is very useful in making the compiler do what I want. It turns out that `fieldtype` was also not considered pure but wrapping it made all optimizations happen 🙂

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [February 15, 2021, 4:21pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/19 "2021-02-15T16:21:52Z")

</div>

@marius311 suggestion was not that you used `Base.@pure` and probably you should not be using it. See the [Base.@pure documentation](https://docs.julialang.org/en/v1/base/base/#Base.@pure). If I am not wrong just calling a function that may be extended by others (or is automatically extended in some cases) make the function not eligible for `@pure`. Unfortunately we would need to bother someone like @JeffreySarnoff to get a better idea if `fieldtype` should be wrapped with `@pure` (I think it would already be, if this was the case, as it is a built-in function)

---

<div class="post-metadata">

**Author:** ![Guillaume\_Leclerc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guillaume_leclerc/32/21940_2.png) [@Guillaume\_Leclerc](https://discourse.julialang.org/u/Guillaume_Leclerc)\
**Post date:** [February 15, 2021, 4:28pm UTC](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278/20 "2021-02-15T16:28:10Z")

</div>

How would I do this then without `@pure`:

```julia
function myInit(T, s::Symbol)
    tpe = fieldtype(T, s)
    zero(tpe)
end

```

With it the llvm is what I expect: a constant  
Without it’s so nasty I don’t even feel like understanding what is going on (and also it just end up returning a constant)

[Next page](https://discourse.julialang.org/t/efficient-reflection-on-structs/55278.md?page=2)