# Compilation latency with constant propagation and unreachable branches

**URL:** <https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667>\
**Category:** Performance\
**Tags:** compilation, ttfx\
**Created:** [May 24, 2024, 8:59am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667 "2024-05-24T08:59:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [May 24, 2024, 8:59am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/1 "2024-05-24T08:59:17Z")

</div>

With

```julia
julia> function f(a, b)
           if a == 1
               return b * b
           elseif a == 2
               return b * b'
           elseif a == 3
               return b' * b
           else
               return b' * b'
           end
       end
f (generic function with 1 method)

julia> g(f::F, b) where {F} = f(1, b)
g (generic function with 1 method)

julia> f2(a, b) = b * b
f2 (generic function with 1 method)

julia> using LinearAlgebra

julia> A = rand(2,2);

```

if we run in separate sessions:

```julia
julia> @time g(f, A);
  1.482260 seconds (3.43 M allocations: 225.566 MiB, 12.40% gc time, 100.00% compilation time)

julia> @time g(f2, A);
  1.321378 seconds (2.34 M allocations: 152.849 MiB, 14.46% gc time, 100.00% compilation time)

julia> VERSION
v"1.10.3"

```

we find that the latter has a lower latency and significantly lower allocations. They’re both compiled to the same code and the dead branches are eliminated in `g(f, A)`, but the constant-propagation appears to add quite a bit of overhead. I wonder if there’s a way to make the TTFX for the first case comparable to the second?

Ideally, I wouldn’t want to change the signatures or use static numbers.

---

<div class="post-metadata">

**Author:** ![aviatesk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aviatesk/32/7610_2.png) [@aviatesk](https://discourse.julialang.org/u/aviatesk)\
**Post date:** [May 24, 2024, 9:34am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/2 "2024-05-24T09:34:26Z")

</div>

Constant-propagation is an additional analysis in inference, so it adds overhead to the compiler. However, constant-propagation can also provide better return types and effects, which can ultimately reduce the overall compile latency of the call graph.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [May 24, 2024, 11:16am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/3 "2024-05-24T11:16:42Z")

</div>

I wonder if this is a case of us not doing enough constant propagation and us inferring the dead branches, instead of disregarding them.

But we would need to check the inference graph with snoop compile.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [May 24, 2024, 11:17am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/4 "2024-05-24T11:17:35Z")

</div>

> [@jishnub](#):
>
> […] the constant-propagation appears to add quite a bit of overhead. I wonder if there’s a way to make the TTFX for the first case comparable to the second?

Are you asking for improvements to the compiler or do want something else?

Tangentially: Is there a simple way of profiling what the compiler does? To get information about what stage takes time, causes allocations etc.?

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [May 24, 2024, 11:49am UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/5 "2024-05-24T11:49:25Z")

</div>

Yes, I am looking for improvements to the compiler if possible, and I suspect exactly what Valentin suggested above: that the dead branches are being inferred. This is because, in [Split generic\_matmul for strided matrices into two halves by jishnub · Pull Request #54552 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/54552), I find that removing a dead branch significantly reduced latency, which I had not expected.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [May 24, 2024, 12:26pm UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/6 "2024-05-24T12:26:51Z")

</div>

> [@abraemer](#):
>
> Tangentially: Is there a simple way of profiling what the compiler does? To get information about what stage takes time, causes allocations etc.?

Not really, for this particular question one could use [Snooping on inference: @snoopi\_deep · SnoopCompile](https://timholy.github.io/SnoopCompile.jl/dev/snoopi_deep/) to get the cost of the inference graph

---

<div class="post-metadata">

**Author:** ![aviatesk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aviatesk/32/7610_2.png) [@aviatesk](https://discourse.julialang.org/u/aviatesk)\
**Post date:** [May 24, 2024, 1:40pm UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/8 "2024-05-24T13:40:54Z")

</div>

> [@abraemer](#):
>
> Tangentially: Is there a simple way of profiling what the compiler does? To get information about what stage takes time, causes allocations etc.?

For benchmarking, you can use:

```julia
using BaseBenchmarks
BaseBenchmarks.load!("inference")
@benchmark BaseBenchmarks.InferenceBenchmarks.@inf_call f(args...)

```

However, please note that this does not necessarily correspond to real-world compile latency. `@inf_call` only compiles the statically analyzable call graph, so it cannot measure the cost of compilation associated with dynamic calls that occur in reality.

---

<div class="post-metadata">

**Author:** ![aviatesk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aviatesk/32/7610_2.png) [@aviatesk](https://discourse.julialang.org/u/aviatesk)\
**Post date:** [May 24, 2024, 2:44pm UTC](https://discourse.julialang.org/t/compilation-latency-with-constant-propagation-and-unreachable-branches/114667/9 "2024-05-24T14:44:46Z")

</div>

> [@jishnub](#):
>
> Yes, I am looking for improvements to the compiler if possible, and I suspect exactly what Valentin suggested above: that the dead branches are being inferred. This is because, in [Split generic\_matmul for strided matrices into two halves by jishnub · Pull Request #54552 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/54552), I find that removing a dead branch significantly reduced latency, which I had not expected.

(I posted this on GitHub as well, but I’ll reiterate it here just in case)

To explain briefly, Julia’s inference works as follows:

For a method `m`:

1. Regular inference: First, inference is performed without constant information.
2. Constant inference: Next, inference is performed with constant information for the same `m`.

These steps are repeated recursively for calls within `m`.

If a method `m` contains a branch that becomes dead when its arguments are specific constant values, regular inference in step 1 cannot use that constant information, so it has to infer including that dead branch. This dead branch is identified as dead by constant inference in step 2, and the final generated code is optimized, but we already paid the compilation latency from step 1.

Now you might wonder why we perform regular inference. The reason is that regular inference can be used for any constant values and also even when there is no constant information, making it highly reusable and cache-beneficial.

Therefore, it is difficult to reduce the latency of constant inference for code that can only be optimized by constant inference. A solution, as implemented in this PR, is to refactor the code so that regular inference can recognize dead branches. By splitting methods into cases and using method dispatch to prevent the inference of dead cases, latency can be improved.