# Reduce compilation time by avoiding specialization

**URL:** <https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493>\
**Category:** Internals & Design\
**Created:** [October 29, 2022, 3:52pm UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493 "2022-10-29T15:52:09Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [October 29, 2022, 3:52pm UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493/1 "2022-10-29T15:52:09Z")

</div>

I’m trying to speed up function tracing in [Umlaut.jl](https://github.com/dfdx/Umlaut.jl). According to the profiler, about 30% of time is spent in a generic function `mkcall()`, which records the call, does some argument transformation and then simply invokes the original function:

```julia
function mkcall(fn::Any, args::Vararg{Any}; kwargs...)
    fn_, args_ = ...
    return fn_(args_...)
end

```

Within `mkcall()` invokation, most time is spent in abstract interpretation & type inference. If I understand it correctly, this is due to Julia compiler specializing `mkcall()` for each combination of function and arguments:

```julia
julia> using MethodAnalysis

julia> methodinstances(mkcall)
7-element Vector{Core.MethodInstance}:
 MethodInstance for Umlaut.mkcall(::typeof(_getfield), ::Variable, ::Int64)
 MethodInstance for Umlaut.mkcall(::typeof(map), ::typeof(unthunk), ::Variable)
 MethodInstance for Umlaut.mkcall(::typeof(tuple), ::Variable, ::Variable)
 MethodInstance for Umlaut.mkcall(::Function, ::Variable, ::Vararg{Variable})
 MethodInstance for Umlaut.mkcall(::Function, ::Function, ::Vararg{Any})
 MethodInstance for Umlaut.mkcall(::typeof(tuple), ::Vararg{Any})
 MethodInstance for Umlaut.mkcall(::Function, ::Variable, ::Vararg{Any})

```

So I tried to avoid excessive compilation using `@nospecialize` as well as turning off inlining and constant propagation as suggested [here](https://discourse.julialang.org/t/why-doesnt-nospecialize-work-in-this-example/88695/7):

```julia
@noinline Base.@constprop :none function mkcall(fn::Any, args::Vararg{Any}; kwargs...)
    @nospecialize
    fn_, args_ = ...
    return fn_(args_...)
end

```

If I then invoke `mkcall` a few times manually, everything works as expected and Julia generates only one specialization. But when I run it on a real case (specifically, tracing `Metalhead.ResNet(18)`), I still get a lot of method instances:

```julia
julia> methodinstances(mkcall)
6-element Vector{Core.MethodInstance}:
 MethodInstance for Umlaut.mkcall(::typeof(_getfield), ::Variable, ::Int64)
 MethodInstance for Umlaut.mkcall(::Any, ::Any)
 MethodInstance for Umlaut.mkcall(::typeof(map), ::typeof(unthunk), ::Variable)
 MethodInstance for Umlaut.mkcall(::typeof(tuple), ::Variable, ::Variable)
 MethodInstance for Umlaut.mkcall(::typeof(tuple), ::Vararg{Any})
 MethodInstance for Umlaut.mkcall(::Any, ::Any, ::Vararg{Any})

```

Why `@nospecialize` doesn’t work in this case? Is there a better way to speed compilation?

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [October 29, 2022, 5:59pm UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493/2 "2022-10-29T17:59:26Z")

</div>

Would the `inferencebarrier` trick described in [https://github.com/JuliaLang/julia/pull/41931#issuecomment-902545562](https://github.com/JuliaLang/julia/pull/41931#issuecomment-902545562) and [Why doesn't `@nospecialize` work in this example? - #6 by tim.holy](https://discourse.julialang.org/t/why-doesnt-nospecialize-work-in-this-example/88695/6) work?

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [October 29, 2022, 8:44pm UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493/3 "2022-10-29T20:44:24Z")

</div>

Somehow I overlooked that comment in the thread, thanks for bringing my attention to it!

I wrapped all arguments in all invocations of `mkcall` with `Base.inferencebarrier()` like this:

```julia
mkcall(Base.inferencebarrier(v_fargs)...; line=Base.inferencebarrier(line))

```

and this:

```julia
mkcall(
    Base.inferencebarrier(getindex),
    Base.inferencebarrier(v),
    Base.inferencebarrier(i);
    line=Base.inferencebarrier("text comment"))
)

```

But don’t see any effect. Not sure this is the correct usage though as the function is undocumented.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [October 30, 2022, 7:46am UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493/4 "2022-10-30T07:46:33Z")

</div>

Use `inferencebarrier` in the _caller(s)_ of `mkcall`. The idea is to prevent inference from knowing the types of the arguments of `mkcall` so that it can’t infer-specialize.

An alternative might be to put `mkcall` in a separate module for which you’ve set `Base.Experimental.@compiler_options`, but I have not experimented with that enough to predict whether that would avoid the need for `inferencebarrier` in the caller. If you try it, let us know the result.

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [October 30, 2022, 12:36pm UTC](https://discourse.julialang.org/t/reduce-compilation-time-by-avoiding-specialization/89493/5 "2022-10-30T12:36:35Z")

</div>

> [@tim.holy](#):
>
> Use `inferencebarrier` in the _caller(s)_ of `mkcall`. The idea is to prevent inference from knowing the types of the arguments of `mkcall` so that it can’t infer-specialize.

I changed all occurrences like this:

```julia
function do_something()
    ...
    mkcall(v_fargs...)
end

```

to this:

```julia
function do_something()
    ...
    mkcall(Base.inferencebarrier(v_fargs)...)
end

```

Is it what you mean by using `Base.inferencebarrier()` in the caller? If so, even after this change I observe multiple method instances for `mkcall`. However, since I’m testing it on a pretty complex example, I suspect it’s some weird corner case that I overlooked despite my best effort.

* * *

However, your second suggestion is brilliant! Adding

```julia
Base.Experimental.@compiler_options optimize=0 compile=min infer=no

```

to the beginning of the module immediately decreased compilation time from 54 to 44 seconds. Now I don’t see `mkcall()` in the flame graph at all, and there are no method instances for it after tracing.

* * *

What’s interesting, I was able to further reduce tracing time to 32 seconds by changing this:

```julia
fargs = (fn, args...)
fargs_ = map_vars(v -> v._op.val, fargs)
fn_, args_ = fargs_[1], fargs_[2:end]
val_ = fn_(args_...)

```

to this:

```julia
fn_ = fn isa V ? fn.op.val : fn
args_ = Any[v isa V ? v.op.val : v for v in args]
val_ = fn_(args_...)

```

Previously, profiler pointed to `fargs_[2:end]` (i.e. `getindex(::Array, ::UnitRange)`) as a major bottleneck.