Function compiles twice but is called just once

General Usage
, , ,
1

Consider the following code:

julia> @generated function f(x)
           @info "Compiling f"
           return :(x + 1)
       end
f (generic function with 1 method)

julia> function type_unstable(y::Vector{Number})
           @info "Running type_unstable"
           x = 2 * y[1]
           return f(x)
       end
type_unstable (generic function with 1 method)

julia> type_unstable(Number[1.0])
[ Info: Compiling f # Wait, the call to `f` is a dynamic dispatch, why are we compiling `f` before knowing the type of `x`?
[ Info: Running type_unstable
[ Info: Compiling f
3.0

julia> using MethodAnalysis

julia> methodinstances(f)
2-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Any) # Why is this `MethodInstance` created?
 MethodInstance for f(::Float64)

In this code, f gets called once, but it gets compiled twice—once with an input of Any and once with an input of Float64.

Why does f get compiled twice, and why is one of those times before even knowing the type of the input?

Also, is there a way to modify this example so the dispatch is still dynamic but it only compiles once?

2

Try this:

julia> @generated function f(x)
           x
           @info "Compiling f"
           return :(x + 1)
       end
f (generic function with 1 method)

julia> function type_unstable(y::Vector{Number})
           @info "Running type_unstable"
           x = 2 * y[1]
           return f(x)
       end
type_unstable (generic function with 1 method)

julia> type_unstable(Number[1.0])

It might have something to do with that x isn’t used in f(x).

1 Like
3

Thanks, that does help me have a cleaner example. But even with this I still see two method instances for f:

julia> @generated function f(x)
           x
           @info "Compiling f"
           return :(x + 1)
       end
f (generic function with 1 method)

julia> function type_unstable(y::Vector{Number})
           @info "Running type_unstable"
           x = 2 * y[1]
           return f(x)
       end
type_unstable (generic function with 1 method)

julia> type_unstable(Number[1.0])
[ Info: Running type_unstable
[ Info: Compiling f # Good, only compiling once as expected.
3.0

julia> using MethodAnalysis

julia> methodinstances(f)
2-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Any) # Wait, why are there two `MethodInstance`s?
 MethodInstance for f(::Float64)

Even though f seemingly was compiled just once, there are still two MethodInstances. Does the f(::Any) method instance have to do with the fact that f is a @generated function?

Maybe I need to update my mental model of @generated functions as follows: A @generated function is really two functions: one that creates the code that runs, and another that runs the generated code. That would explain why there are two method instances, the ::Any instance being the method that creates the generated code. Is this mental model correct?

But why is there only one Compiling f info statement that prints in this version of f?

4

Don’t take my comments for granted because these compiler topics are beyond my understanding, but I will try to comment on some of the details I have noticed.

Even though f seemingly was compiled just once, there are still two MethodInstances. Does the f(::Any) method instance have to do with the fact that f is a @generated function?

About the number of instances, I would say it has to do with the fact that Number is not a concrete type and not with the fact that it is a generated function.

julia> isconcretetype(Number)
false

I say this because if you define the type_unstable function without any type annotation and you run the function for concrete types, no f(::Any) function is generated.

julia> @generated function f(x)
           @info "Compiling f"
           return :(x + 1)
       end
f (generic function with 1 method)

julia> function type_unstable(y)
           @info "Running type_unstable"
           x = 2 * y[1]
           return f(x)
       end
type_unstable (generic function with 1 method)

julia> using MethodAnalysis

julia> type_unstable([1.0])
[ Info: Compiling f
[ Info: Running type_unstable
3.0

julia> methodinstances(f)
1-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Float64)

julia> type_unstable([1])
[ Info: Compiling f
[ Info: Running type_unstable
3

julia> methodinstances(f)
2-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Float64)
 MethodInstance for f(::Int64)

julia> type_unstable([ComplexF32(1.0)])
[ Info: Compiling f
[ Info: Running type_unstable
3.0f0 + 0.0f0im

julia> methodinstances(f)
3-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Float64)
 MethodInstance for f(::Int64)
 MethodInstance for f(::ComplexF32)

julia> type_unstable(Number[1.0])
[ Info: Compiling f
[ Info: Running type_unstable
3.0

julia> type_unstable(Number[1.0])
methodinstances(f)
4-element Vector{Core.MethodInstance}:
 MethodInstance for f(::Float64)
 MethodInstance for f(::Int64)
 MethodInstance for f(::ComplexF32)
 MethodInstance for f(::Any)

But why is there only one Compiling f info statement that prints in this version of f?

It may (or may not) have something to do with something mentioned in the manual. From https://docs.julialang.org/en/v1/manual/metaprogramming/:

The number of times a generated function is generated might be only once, but it might also be more often, or appear to not happen at all. As a consequence, you should never write a generated function with side effects - when, and how often, the side effects occur is undefined…

2 Likes
5

Interesting, I hadn’t noticed that.

And thanks for reminding me about the docs, it had been a while since I read the section on generated functions, and that satisfactorily answered my questions.

1 Like