# Why does this broadcast operation require specialization for optimal performance?

**URL:** <https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547>\
**Category:** Performance\
**Tags:** broadcast, array\
**Created:** [February 13, 2023, 10:12am UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547 "2023-02-13T10:12:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [February 13, 2023, 10:12am UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/1 "2023-02-13T10:12:19Z")

</div>

```julia
julia> g1(f, x, y, z) = (y .= f.(x, z); y);

julia> g2(f, x, y, z) = (broadcast!(f, y, x, z); y);

julia> g3(f::F, x, y, z) where {F} = (y .= f.(x, z); y);

julia> x = rand(100); y = zeros(100); z = rand(100);

julia> @btime g1(*, $x, $y, $z);
  231.065 ns (2 allocations: 64 bytes)

julia> @btime g2(*, $x, $y, $z);
  23.848 ns (0 allocations: 0 bytes)

julia> @btime g3(*, $x, $y, $z);
  28.712 ns (0 allocations: 0 bytes)

julia> g1(*, x, y, z) == g2(*, x, y, z) == g3(*, x, y, z)
true

julia> VERSION
v"1.9.0-beta3"

```

I don’t see any dynamic dispatch in `g1`:

```julia
julia> @report_opt g1(*, x, y, z)
No errors detected

```

Why does `g1` require specializing on the function for optimal performance?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [February 13, 2023, 1:41pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/2 "2023-02-13T13:41:49Z")

</div>

see the performance tips. functions by default (unlike almost everything else) aren’t specialized on by default.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [February 13, 2023, 1:50pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/3 "2023-02-13T13:50:03Z")

</div>

It _is_ confusing though. The performance tips say that specializing only happens when the function or type is “used” (I presume to mean a call) and not when it is passed as an argument to a higher order function, so I would not expect `g2` to specialize on `f`, I would expect `g2` to perform the same as `g1`.

It’s also strange that `@report_opt` suggests no dynamic dispatch in `g1` when that’s what no-specialization would do. The allocations and performance seems to suggest it.

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [February 13, 2023, 3:39pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/4 "2023-02-13T15:39:57Z")

</div>

Despite being very familiar with [that section of the Performance Tips](https://docs.julialang.org/en/v1/manual/performance-tips/#Be-aware-of-when-Julia-avoids-specializing), I frequently fail to accurately predict when it will and won’t apply. I’m not sure that it is entirely clear or accurate. Next time I catch a case with unexpected behavior I’ll be sure to post about it, but it seems this thread already has an example where it’s either misleading or otherwise requires understanding the nuance of broadcast lowering.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [February 13, 2023, 4:30pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/5 "2023-02-13T16:30:15Z")

</div>

Perhaps I’m starting to understand now. Looking into what `g2` is calling, I see

```julia
broadcast!(f::Tf, dest, As::Vararg{Any,N}) where {Tf,N} = (materialize!(dest, broadcasted(f, As...)); dest)

```

so `broadcast!` forces a specialization on the function before passing it to `broadcasted`, whereas `y .= x .* z` lowers to `broadcasted`, and simply passes the function to it without specializing on it. As for why this makes a difference, I’m not certain, since it should ultimately be passed down to `Broadcasted` in either case, which does specialize on the function.

If this is indeed the issue, perhaps the specialization should happen in `broadcasted` instead, which would handle all cases?

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [February 14, 2023, 11:51am UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/6 "2023-02-14T11:51:16Z")

</div>

> [@jishnub](#):
>
> lowers to `broadcasted`, and simply passes the function to it without specializing on it

There is a lot of `@inline`ing going on in broadcast.jl, and I’m not exactly sure if the chain of function calls are just inlined to the point where only specializing function calls remain. I do suspect that because of `g3`’s performance; even if `g3` itself specializes, if `y .= f.(x, z)` doesn’t then I expect runtime dispatch.

Which makes me speculate this is some inlining into the `@btime` loop going on? But I don’t know of a way to simply inspect which function calls are inlineable.

---

<div class="post-metadata">

**Author:** ![N5N3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/n5n3/32/17663_2.png) [@N5N3](https://discourse.julialang.org/u/N5N3)\
**Post date:** [February 14, 2023, 1:02pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/7 "2023-02-14T13:02:42Z")

</div>

[broadcast: disable nospecialize logic for outer method signature by vtjnash · Pull Request #43200 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/43200) would reduce the possiblity of similar problem.  
The extensive `@inline` does make life worse in many places. (As a non-inlined `materialize!` would always be specialized)

Of course, if we remove all `@inline` then the const propagation of broadcast would be turned off,  
which means that `y .= 2 .* x` and `f(x) = 2x; y .= f.(x)` might have different performance.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [February 15, 2023, 3:08pm UTC](https://discourse.julialang.org/t/why-does-this-broadcast-operation-require-specialization-for-optimal-performance/94547/8 "2023-02-15T15:08:00Z")

</div>

Indeed, in this case, adding call-site inlining seems to remove the disparity:

```julia
julia> @btime @inline g1(*, $x, $y, $z);
  21.322 ns (0 allocations: 0 bytes)

julia> @btime g1(*, $x, $y, $z);
  213.411 ns (2 allocations: 64 bytes)

```