# Inlining behavior: constant prop. fails, I need help in understanding "why"

**URL:** <https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134>\
**Category:** Performance\
**Created:** [November 4, 2018, 9:07am UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134 "2018-11-04T09:07:48Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 4, 2018, 9:07am UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/1 "2018-11-04T09:07:48Z")

</div>

MWE:

```julia
using StaticArrays, BenchmarkTools
struct DelayEmbedding{D}
    delays::SVector{D, Int}
end

@inline DelayEmbedding(D::Int) = DelayEmbedding(Val{D}())
@inline function DelayEmbedding(::Val{D}) where {D}
    idxs = [k for k in 1:D]
    return DelayEmbedding{D}(SVector{D, Int}(idxs...))
end

@generated function (r::DelayEmbedding{D})(s::AbstractArray{T}, i) where {D, T}
    gens = [:(s[i + r.delays[$k]]) for k=1:D]
    quote
        @inbounds return SVector{$D+1,T}(s[i], $(gens...))
    end
end

```

This works and does not allocate:

```julia
x = rand(10000);
e = DelayEmbedding(1)
@btime $e($x, 4);
  # 2.052 ns (0 allocations: 0 bytes)

```

Buuut, when I create a minimal wrapper function that instantiates `DelayEmbedding`, I get allocations:

```julia
@inline function reconstruct(s::AbstractVector{T}, D::Int) where {T}
  de::DelayEmbedding{D} = DelayEmbedding(Val{D}())

  c = 0.0
  for i in 1:100
    data = de(s, i)
    c += data[1]
  end

  return c
end

```

I now benchmark this `reconstruct` function:

```julia
D = 1
@btime reconstruct($x, $D);
  # 8.348 μs (304 allocations: 6.39 KiB)

@btime reconstruct($x, 1);
  # 736.273 ns (2 allocations: 112 bytes)

```

It is “weird” that if `D` is a literal the performance difference is massive…

* * *

To my understanding, it is possible to write such code as above in Julia 1.0 because of constant propagation. On the other hand it is obvious that I have not understood how it works, since my code fails.

Is there any way to make my code be as fast as in the second case, even when giving in a variable instead of literal?

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [November 4, 2018, 9:57am UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/2 "2018-11-04T09:57:10Z")

</div>

Your code is not copy-pasteable (first two blocks: `using StaticArrays, BenchmarkTools`, third block: globals tau and D).

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 4, 2018, 12:37pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/3 "2018-11-04T12:37:32Z")

</div>

Thanks, I’ve edited the code. I have also removed everything irrelevant, like the parameter `τ`. What is left is the least amount of code I need to replicate my problem.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 4, 2018, 3:21pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/4 "2018-11-04T15:21:27Z")

</div>

This is how I usually benchmark constant propagation:

```julia
julia> function f(x)
           D = 1
           reconstruct(x, D)
       end
f (generic function with 1 method)

julia> @btime f($x)
  871.847 ns (2 allocations: 112 bytes)
50.48725909006644

```

---

<div class="post-metadata">

**Author:** ![Non-Contradiction](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-contradiction/32/2208_2.png) [@Non-Contradiction](https://discourse.julialang.org/u/Non-Contradiction)\
**Post date:** [November 4, 2018, 3:39pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/5 "2018-11-04T15:39:23Z")

</div>

The global variable is not a constant unless making it a constant,

```julia
const DD = 1;
@btime reconstruct($x, DD);

```

works for me.

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 7, 2018, 11:09am UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/6 "2018-11-07T11:09:27Z")

</div>

@kristoffer.carlsson so if I understood correctly, constant propagation does not work the way I thought. I thought it was about some values that are constant between functions and/or local scopes and are understood as “some kind of literals”. The value `D` in my case is e.g. coming from a for loop,

```julia
for D in [1, 2]
   @btime reconstruct(x, D)
end

```

Benchmarking the above I still don’t get the “good” performance, I only get the “bad” one.

So, to conclude, it is not possible to get the “good” performance if D is not a literal? In your above example `D = 1` inside a function is equivalent with it being a literal.

@Non-Contradiction I don’t really know what you are talking about…

```julia
julia> const f = 2
2

julia> @btime reconstruct($x, $f);
  9.813 μs (305 allocations: 6.47 KiB)

```

gives the bad performance, not the good one.

---

<div class="post-metadata">

**Author:** ![Non-Contradiction](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-contradiction/32/2208_2.png) [@Non-Contradiction](https://discourse.julialang.org/u/Non-Contradiction)\
**Post date:** [November 7, 2018, 1:58pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/7 "2018-11-07T13:58:27Z")

</div>

Please pay attention, that in my code, `@btime reconstruct($x, DD);` is used, not `@btime reconstruct($x, $DD);`, and `@btime reconstruct($x, DD);` gives good performance, because `DD` is a constant already, so no need to use `$DD`, just as there is no need of `$` before function names. But I don’t know what happens with `$` before `DD`, it gives bad performance for me too.

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 7, 2018, 2:41pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/8 "2018-11-07T14:41:34Z")

</div>

Sorry, I overlooked this! Thanks for clarifying and illustrating! 🙂

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [November 7, 2018, 4:35pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/9 "2018-11-07T16:35:34Z")

</div>

Well, const-prop should not be relied upon anyway so the correct spelling of your construction could be

```julia
julia> @noinline function reconstruct_inner(s::AbstractVector{T}, de) where {T}
         c = 0.0
         for i in 1:100
           data = de(s, i)
           c += data[1]
         end

         return c
         end
julia> @noinline reconstruct_outer(s,D)=reconstruct_inner(s, DelayEmbedding(D))

julia> x = rand(10000); D=2;
julia> @btime reconstruct_outer($x,$D);
  6.846 μs (5 allocations: 208 bytes)

julia> @btime reconstruct($x,$D);
  14.249 μs (305 allocations: 6.47 KiB)

julia> de = DelayEmbedding(D);
julia> @btime reconstruct_outer($x,$D);
  5.661 μs (5 allocations: 208 bytes)

julia> de = DelayEmbedding(D);

julia> @btime reconstruct_inner($x,$de);
  112.078 ns (0 allocations: 0 bytes)

julia> @btime reconstruct($x,$D);
  13.708 μs (305 allocations: 6.47 KiB)

julia> @btime reconstruct($x,$2);
  13.818 μs (305 allocations: 6.47 KiB)

julia> @btime reconstruct($x,2);
  835.253 ns (3 allocations: 160 bytes)

```

In other words: You problem is that you switch between value-domain and type-domain, which const-prop can sometimes eliminate, but sometimes not. Just construct your delay-embedding in some outer function, make a `@noinline boundary to an inner function where the delay embedding is fixed and its type is part of the signature, and be happy.

If you can get away with this, consider

```julia
julia> struct DelayEmbedding2{D} end
julia> de2=DelayEmbedding2{D}();

julia> @generated function (r::DelayEmbedding2{D})(s::AbstractArray{T}, i) where {D, T}
           gens = [:(s[i + $k]) for k=1:D]
           quote
               @inbounds return SVector{$D+1,T}(s[i], $(gens...))
           end
           end

```

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 8, 2018, 4:20pm UTC](https://discourse.julialang.org/t/inlining-behavior-constant-prop-fails-i-need-help-in-understanding-why/17134/10 "2018-11-08T16:20:40Z")

</div>

Thanks for the detailed reply foobar. Independent of your reply I have also come to the same conclusion (that I have to split my top level function), although I didn’t have time to answer before!

Now it is almost as fast as the “fastest” version and this is truly great!