# Strange allocations during broadcasting

**URL:** <https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605>\
**Category:** Performance\
**Created:** [July 21, 2019, 2:05pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605 "2019-07-21T14:05:58Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [July 21, 2019, 2:05pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/1 "2019-07-21T14:05:58Z")

</div>

Can you please explain me why in the following code the second broadcasting results in 3 allocations?

```julia
using BenchmarkTools

u = rand(100)
k1 = similar(u)
k2 = similar(u)
k3 = similar(u)
k4 = similar(u)

@btime @. $u = $u + 1 * ($k1 + $k2 + $k3)
@btime @. $u = $u + 1 * ($k1 + $k2 + $k3 + $k4)
@btime @. $u = $u + ($k1 + $k2 + $k3 + $k4)

```

```julia
  100.137 ns (0 allocations: 0 bytes)
  143.974 ns (3 allocations: 80 bytes)
  127.223 ns (0 allocations: 0 bytes)

```

What is more confusing for me is that if I swap the last two lines, there will be no allocations.  
Thank you.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [March 25, 2020, 1:49pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/2 "2020-03-25T13:49:29Z")

</div>

Up.

Meanwhile, in Julia 1.4 there are already 4 allocations in the above code.  
Just to highlight the problem, the following code

```julia
using BenchmarkTools
# import DifferentialEquations

function foo(u, k1, k2, k3, k4)
    for i=1:1000
        @. u = u + 1.0 * (k1 + k2 + k3 + k4)
    end
    return nothing
end

function main()
    u = rand(100)
    k1 = zero(u)
    k2 = zero(u)
    k3 = zero(u)
    k4 = zero(u)

    @btime foo($u, $k1, $k2, $k3, $k4)
end

main()

```

results in

```julia
155.048 μs (4000 allocations: 93.75 KiB)

```

Surprisingly, if I import DifferentialEquations (uncomment the second line), the result will be

```julia
122.759 μs (0 allocations: 0 bytes)

```

That is, there are no more allocations.

The above code is typical for various Runge-Kutta methods. So far in my realizations of Runge-Kutta methods of order higher than 3 I inevitably see these broadcasting allocations. Can you, please, help me to understand what is going on. Especially, why the import of DifferentialEquations solves the problem.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 25, 2020, 4:05pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/3 "2020-03-25T16:05:20Z")

</div>

It seems there is some heuristic that is getting hit because removing the `+k4` makes it stop allocating.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [March 25, 2020, 4:33pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/4 "2020-03-25T16:33:49Z")

</div>

Do you have any ideas where I can start to dig in?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 25, 2020, 5:20pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/5 "2020-03-25T17:20:43Z")

</div>

Doing

```julia
julia> function foo(u, k1, k2, k3, k4)
           @. u = u + 1.0 * (k1 + k2 + k3 + k4)
           return nothing
       end
foo (generic function with 2 methods)

julia> @code_warntype optimize=true foo(u, k1, k2, k3, k4)

```

I see a

```julia
8 ─── %24 = invoke Base.Broadcast.combine_axes(1.0::Float64, %2::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(+),NTuple{4,Array{Float64,1}}})::Tuple{Union{Base.OneTo{Int64}, UnitRange{Int64}},Vararg{Any,N} where N}

```

which is not inferred to a concrete type. That’s the place I would probably start poking at.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 25, 2020, 5:31pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/6 "2020-03-25T17:31:20Z")

</div>

> [@fedoroff](#):
>
> The above code is typical for various Runge-Kutta methods. So far in my realizations of Runge-Kutta methods of order higher than 3 I inevitably see these broadcasting allocations. Can you, please, help me to understand what is going on. Especially, why the import of DifferentialEquations solves the problem.

Probably due to our broadcast fixes, i.e. [https://github.com/SciML/DiffEqBase.jl/blob/master/src/diffeqfastbc.jl#L18-L19](https://github.com/SciML/DiffEqBase.jl/blob/master/src/diffeqfastbc.jl#L18-L19)

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 25, 2020, 5:48pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/7 "2020-03-25T17:48:29Z")

</div>

Do you have a PR that upstreams those?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 25, 2020, 5:59pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/8 "2020-03-25T17:59:16Z")

</div>

There wasn’t before, but [https://github.com/JuliaLang/julia/pull/35260](https://github.com/JuliaLang/julia/pull/35260) is it. Someone should probably double check that those changes are sufficient to get rid of these allocations.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [March 25, 2020, 5:59pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/9 "2020-03-25T17:59:32Z")

</div>

It seems this is not the case:

```julia
using BenchmarkTools

import Base.Broadcast: Broadcasted
@inline combine_axes(A, B) = broadcast_shape(broadcast_axes(A), broadcast_axes(B))
@inline check_broadcast_axes(shp, A::Union{Number, Array, Broadcasted}) = check_broadcast_shape(shp, axes(A))

function foo(u, k1, k2, k3, k4)
    for i=1:1000
        @. u = u + 1.0 * (k1 + k2 + k3 + k4)
    end
    return nothing
end

function main()
    u = rand(100)
    k1 = zero(u)
    k2 = zero(u)
    k3 = zero(u)
    k4 = zero(u)

    @btime foo($u, $k1, $k2, $k3, $k4)
end

main()

```

```julia
177.050 μs (4000 allocations: 93.75 KiB)

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 25, 2020, 6:02pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/10 "2020-03-25T18:02:26Z")

</div>

> [@fedoroff](#):
>
> import Base.Broadcast: Broadcasted @inline combine\_axes(A, B) = broadcast\_shape(broadcast\_axes(A), broadcast\_axes(B)) @inline check\_broadcast\_axes(shp, A::Union{Number, Array, Broadcasted}) = check\_broadcast\_shape(shp, axes(A))

You’re shadowing and not overloading:

```julia
@inline Base.Broadcast.combine_axes(A, B) = broadcast_shape(broadcast_axes(A), broadcast_axes(B))
@inline Base.Broadcast.check_broadcast_axes(shp, A::Union{Number, Array, Broadcasted}) = check_broadcast_shape(shp, axes(A))

```

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [March 25, 2020, 6:08pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/11 "2020-03-25T18:08:06Z")

</div>

Thank you. Now it works:

```julia
using BenchmarkTools

@inline Base.Broadcast.combine_axes(A, B) = Base.Broadcast.broadcast_shape(Base.Broadcast.broadcast_axes(A), Base.Broadcast.broadcast_axes(B))

function foo(u, k1, k2, k3, k4)
    for i=1:1000
        @. u = u + 1.0 * (k1 + k2 + k3 + k4)
    end
    return nothing
end

function main()
    u = rand(100)
    k1 = zero(u)
    k2 = zero(u)
    k3 = zero(u)
    k4 = zero(u)

    @btime foo($u, $k1, $k2, $k3, $k4)
end

main()

```

```julia
136.010 μs (0 allocations: 0 bytes)

```

If it is not hard, can you give a brief explanation on what this patch do?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 25, 2020, 6:15pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/12 "2020-03-25T18:15:51Z")

</div>

Splatting can sometimes cause a tuple to allocate if everything doesn’t inline or the world doesn’t align its chakras correctly. This makes it so that way a common case doesn’t ever splat, so boom now it can’t be a problem.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [March 25, 2020, 6:18pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/13 "2020-03-25T18:18:08Z")

</div>

OK. Thank you.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 25, 2020, 6:20pm UTC](https://discourse.julialang.org/t/strange-allocations-during-broadcasting/26605/14 "2020-03-25T18:20:45Z")

</div>

I find it’s easiest to just do `@eval Base.Broadcast begin ... end` to paste in these sorts of patches interactively without editing. You can also use Revise, but that can be trickier and isn’t as obvious.