Mapping a 'getter'

Could you name which one you had in mind, I can’t guess.

The nice thing beyond just visual distinction is that a macro and function with shared purpose can share a name e.g. @eval/eval. I don’t think it’s as simple as choosing to require an @ character, it’s fairly necessary because function calls and macro calls look very different and source code looks very different from Expr. I don’t know if anyone still considers Julia homoiconic but it’s definitely not on the level of Lisp.

Common Lisp?

I don’t have a Julia REPL handy and maybe I’m mis-remembering, but isn’t there a syntax feature that automatically interprets _.age as the equivalent of an anonymous function p -> p.age? If I remember correctly, it only currently works for accessing fields and indices (e.g. _[1]). This would enable the above to be written simply as map(_.age, ps).

All-underscore variables don’t work in right-hand expressions, though it’s not an outright invalid expression e.g. :(_[]) is valid while :(for end) is not. Macros can transform right-hand underscores, like Pipe.jl.

Type instability is the problem.

julia> getfirst(z) = getfield.(z,:first)
getfirst (generic function with 1 method)

julia> z = [rand(1:10) => rand() for _ in 1:1000];

julia> using BenchmarkTools

julia> @btime getfirst($z);
  22.500 μs (1002 allocations: 39.20 KiB)

julia> @code_warntype getfirst(z)
MethodInstance for getfirst(::Vector{Pair{Int64, Float64}})
  from getfirst(z) @ Main REPL[207]:1
Arguments
  #self#::Core.Const(getfirst)
  z::Vector{Pair{Int64, Float64}}
Body::Union{Vector{Float64}, Vector{Int64}, Vector{Real}}
1 ─ %1 = Base.broadcasted(Main.getfield, z, :first)::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(getfield), Tuple{Vector{Pair{Int64, Float64}}, Base.RefValue{Symbol}}}
│   %2 = Base.materialize(%1)::Union{Vector{Float64}, Vector{Int64}, Vector{Real}}
└──      return %2

julia> @btime (x -> x.first).($z);
  574.302 ns (1 allocation: 7.94 KiB)

julia> @btime getfield.($z,:first); # just to compare with the previous
  22.500 μs (1002 allocations: 39.20 KiB)

Notice the non-concrete return type and the type instability this function introduces. Const-prop does not always work well through broadcast, so broadcasting getfield/getproperty can often introduce type instability that a dedicated accessor does not.

Of course, the initial example had a struct with untyped fields so instability was inevitable there. But here I’ve shown that broadcasting fails even where it could succeed.

1 Like

I figured out where I read that, in a Julia PR by @stevengj:

RFC: curry underscore arguments to create anonymous functions by stevengj · Pull Request #24990 · JuliaLang/julia · GitHub

Maybe I mis-read the parenthesized sentence in the first paragraph as indicating this part of the functionality was already a thing.

getfield.(z,(:first,)) improves the inference to Union{Vector{Int}, Vector{Real}} and ends up as performant, but the inference isn’t as good as Vector{Int} of the broadcasted anonymous function. A constant literal symbol in the method body really does help the compiler more than a container of symbols. Since getproperty/getfield already exceptionally specializes on constant field names or integers, I imagine this optimization could exist for broadcasted getproperty/getfield?

Now you know how much of an old geezer I am :blush:

If you were talking about Lisp, I can’t find what you’re referring to by automatic generation of accessor methods. The closest I could look up was defclass specifying :reader, :writer, :accessor per slot, but those are manually specified with arbitrary names.

I don’t get it. x.first calls getfield(x, :first), how does the extra level of indirection help?

I think it has to do with typeof(getfield(::T, ::Symbol)) being impossible in the type domain (for heterogeneous field types) because all that is known about the field is that it is ::Symbol. Constant propagation (or concrete evaluation or some other related concept that is subtly different that I wouldn’t identify correctly) of the specific value of the Symbol is necessary to know which field (and type) is being accessed. If sufficient optimization magic occurs, then this all works out. But broadcasting seems to usually be enough to prevent this.

Or maybe I’m quite wrong as to the technical reason. I just know this can be unreliable because I’ve been bit by it before.

Whatever it is, there’s a related issue #43333. There is a comment there that provides a tad more detail, but I’m not qualified to explain it here.

1 Like

I’m not convinced that the problem is only with getfield:

julia> getfirst_scalar(z) = getfield(z, :first);

julia> getfirst_broadcasted(z) = getfield.(z, :first);

julia> @btime getfirst_scalar.($z);
  831.818 ns (1 allocation: 7.94 KiB)

julia> @btime getfirst_broadcasted($z);
  30.800 μs (1002 allocations: 39.20 KiB)

julia> @btime getfirst_broadcasted.($z);  # <- extra dot
  2.278 μs (1 allocation: 7.94 KiB)

This is strange to me. I mean, getifield is the mechanism for accessing fields. Everything builds on this, how can it be slow?

2 Likes

x.first gets lowered into getproperty(x, :first) which then calls getfield(x, :first). We can infer the type returned due to constant propagation.


julia> struct Foo end

julia> f() = Foo().first
f (generic function with 1 method)

julia> @code_lowered f()
CodeInfo(
1 ─ %1 = Main.Foo()
│   %2 = Base.getproperty(%1, :first)
└──      return %2
)

julia> Base.getproperty(::Foo, s::Symbol) = 42

julia> f()
42

julia> @code_warntype f()
MethodInstance for f()
  from f() @ Main REPL[2]:1
Arguments
  #self#::Core.Const(f)
Body::Int64
1 ─ %1 = Main.Foo()::Core.Const(Foo())
│   %2 = Base.getproperty(%1, :first)::Core.Const(42)
└──      return %2

We can even complicate the situation as follows.

julia> Base.getproperty(::Foo, s::Symbol) = s == :first ? 42 :  π

julia> @code_warntype f()
MethodInstance for f()
  from f() @ Main REPL[2]:1
Arguments
  #self#::Core.Const(f)
Body::Int64
1 ─ %1 = Main.Foo()::Core.Const(Foo())
│   %2 = Base.getproperty(%1, :first)::Core.Const(42)
└──      return %2

julia> g() = Foo().the_answer_to_everything_else
g (generic function with 1 method)

julia> @code_warntype g()
MethodInstance for g()
  from g() @ Main REPL[13]:1
Arguments
  #self#::Core.Const(g)
Body::Irrational{:π}
1 ─ %1 = Main.Foo()::Core.Const(Foo())
│   %2 = Base.getproperty(%1, :the_answer_to_everything_else)::Core.Const(π)
└──      return %2
1 Like