# Intermittent type inference calls during method evaluation

**URL:** <https://discourse.julialang.org/t/intermittent-type-inference-calls-during-method-evaluation/82241>\
**Category:** Internals & Design\
**Tags:** compilation, ttfx\
**Created:** [June 4, 2022, 5:07pm UTC](https://discourse.julialang.org/t/intermittent-type-inference-calls-during-method-evaluation/82241 "2022-06-04T17:07:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rikh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rikh/32/204104_2.png) [@rikh](https://discourse.julialang.org/u/rikh)\
**Post date:** [June 4, 2022, 5:07pm UTC](https://discourse.julialang.org/t/intermittent-type-inference-calls-during-method-evaluation/82241/1 "2022-06-04T17:07:53Z")

</div>

When looking through a first call for a longish method via [StatProfilerHTML.jl](https://github.com/tkluck/StatProfilerHTML.jl), I see some type inference happening during the method body evaluation:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/6/f/6ff2680eb7d27b0dba3012a295649f3bf769ba30.png)

(This is in Julia 1.8-rc1.)

I think that this is caused by the method being too long which caused the compiler to give up on inferring the whole method. Is this assumption correct?

My guess is also that storing the methodinstance is aborted once inference takes too long, so to ensure that more methodinstances are stored, I just need to avoid methods where typeinference happens during evaluation of the body? This could explain why some methods don’t “take” if you try to precompile them as mentioned by @timholy in [Consider `generate_precompile` prior to module close · Issue #38951 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/38951#issuecomment-749153888)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 4, 2022, 6:22pm UTC](https://discourse.julialang.org/t/intermittent-type-inference-calls-during-method-evaluation/82241/2 "2022-06-04T18:22:52Z")

</div>

> [@rikh](#):
>
> I think that this is caused by the method being too long which caused the compiler to give up on inferring the whole method. Is this assumption correct?

I don’t think so. It just looks like it’s deferred some type inference due to you working on arbitrary `Expr` and it being potentially beneficial to compile better code for the branch there.

---

<div class="post-metadata">

**Author:** ![rikh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rikh/32/204104_2.png) [@rikh](https://discourse.julialang.org/u/rikh)\
**Post date:** [June 5, 2022, 10:41am UTC](https://discourse.julialang.org/t/intermittent-type-inference-calls-during-method-evaluation/82241/3 "2022-06-05T10:41:14Z")

</div>

I think you’re right about the runtime dispatch, but I’m not so sure about the branch. It doesn’t match what I’ve seen while looking at the profiler. Or do you mean the function which is dispatched into by “compile better code for the branch there”? As far as I understand, LLVM code is either a hit or a miss. Or the complete function body is compiled or it isn’t. There isn’t some partial compilation which is then optimized during runtime.

Based on the following, I would guess that the upper method `explore_call!` is compiled already to LLVM code, but the runtime dispatch isn’t yet. But, before stepping into the runtime dispatch, Julia has to compile that function after running type inference. That’s why the type inference shows up in the `explore_call!` body: Julia hasn’t stepped into the JIT compiled lower function.

```julia
warmup(x) = x
precompile(warmup, (Int,))
@time @eval warmup(1)

function f(x)
    out = Base.inferencebarrier(x)
    return out + 1
end
precompile(f, (Int,))
@time @eval f(1)

function g(x)
    out = Base.inferencebarrier(x)::Int
    return out + 1
end
precompile(g, (Int,))
@time @eval g(1)

h(x) = x + 1
precompile(h, (Int,))
@time @eval h(1)

@noinline sub(x) = x

function k(x)
    out = sub(Base.inferencebarrier(x))
    return out + 1
end
precompile(k, (Int,))
@time @eval k(1)

```

```julia
  0.003000 seconds (911 allocations: 55.395 KiB, 89.77% compilation time)
  0.000149 seconds (41 allocations: 1.906 KiB)
  0.000154 seconds (41 allocations: 1.906 KiB)
  0.000145 seconds (41 allocations: 1.906 KiB)
  0.002851 seconds (434 allocations: 29.125 KiB, 92.99% compilation time)

```

Note in this example that the call `@time @eval k(1)` shows compilation time because `sub` wasn’t compiled by the `precompile` call.

EDIT: I just realize that even when all types are known inside a method, it is still possible that Julia cannot compile the LLVM code yet because the function that is dispatched into has to be compiled still, so that’s why there are separate interpreted and compiled modes. Only once all submethods are compiled, the link to that compiled function can be hardcoded into the LLVM code.