# Why is broadcasting allocating so much memory?

**URL:** <https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874>\
**Category:** Performance\
**Created:** [May 21, 2020, 8:29am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874 "2020-05-21T08:29:25Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [May 21, 2020, 8:29am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/1 "2020-05-21T08:29:26Z")

</div>

In the below dummy code I show that doing the sum over the 4-vectors using broadcasting is an order of magnitude slower than iteration of the 4-vector manually. However, I don’t understand why - and whether I can keep using some form of broadcasting that is just as fast the manual version.

Any ideas?

```julia
using BenchmarkTools

function func1(arr1, arr2)
    for row in axes(arr1, 2)
        for i in axes(arr2, 2)
            @views @. arr1[:, row] = arr1[:, row] + arr2[:, i]
        end
    end
    return arr1
end

function func2(arr1, arr2)
    for row in axes(arr1, 2)
        for i in axes(arr2, 2)
            for j in 1:4
                arr1[j, row] = arr1[j, row] + arr2[j, i]
            end
        end
    end
    return arr1
end

arr1 = zeros(4, 100_000)
arr2 = rand(4, 256)

@btime func1($arr1, $arr2) evals=1 samples=10 seconds=100
# 653.253 ms (76800000 allocations: 3.43 GiB)

@btime func2($arr1, $arr2) evals=1 samples=10 seconds=100
# 73.863 ms (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [May 21, 2020, 8:56am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/2 "2020-05-21T08:56:39Z")

</div>

Just a guess, but it may be because of the created views. On 1.4.1, they still need allocations, but on 1.5 those are gone. I don’t have a build of 1.5 at hand, but if you try it out, that might explain the difference.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 21, 2020, 9:49am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/3 "2020-05-21T09:49:03Z")

</div>

Yeah it’s probably the views causing it. On a different note, I noticed that your second function got quite a bit faster by using `@avx` from LoopVectorization.jl:

```julia
function func2(arr1, arr2)
    @avx for row in axes(arr1, 2)
        for i in axes(arr2, 2)
            for j in 1:4
                arr1[j, row] = arr1[j, row] + arr2[j, i]
            end
        end
    end
    return arr1
end

```

The difference is

```julia
74.175 ms (0 allocations: 0 bytes) # Without @avx
7.488 ms (0 allocations: 0 bytes) # With @avx

```

---

<div class="post-metadata">

**Author:** ![ettersi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ettersi/32/6829_2.png) [@ettersi](https://discourse.julialang.org/u/ettersi)\
**Post date:** [May 21, 2020, 9:50am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/4 "2020-05-21T09:50:56Z")

</div>

I don’t know for sure, but it seems plausible that the culprit is the `@views`. Although much cheaper than copying the data, creating views is not entirely free, and since in your case you create a view for every four FLOP, the costs of creating all these views add up.

I am not sure whether this issue can be fixed. Obviously, the two functions which you posted could be replaced with

```julia
func3(arr1, arr2 ) = ( arr1 .= sum(arr2, dims=2) )

```

but I am assuming the posted code is intended as a MWE for something more complicated. Maybe a similar trick could work there?

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [May 21, 2020, 10:03am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/5 "2020-05-21T10:03:49Z")

</div>

I don’t believe it is the views, as I already tried a similar example as you can see below where _even using views_, so long as I don’t broadcast over the 4-vectors I get 0 allocations.

```julia
function func3(arr1, arr2)
    for row in axes(arr1, 2)
        for i in axes(arr2, 2)
            v1, v2 = view(arr1, :, row), view(arr2, :, i)
            for j in 1:4
                arr1[j, row] = v1[j] + v2[j]
            end
        end
    end
    return arr1
end

@btime func3($arr1, $arr2) evals=1 samples=10 seconds=100
# 73.788 ms (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [May 21, 2020, 10:04am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/6 "2020-05-21T10:04:54Z")

</div>

> [@torrance](#):
>
> I don’t believe it is the views, as I already tried a similar example as you can see below where _even using views_ , so long as I don’t broadcast over the 4-vectors I get 0 allocations.

It’s the combination of allocating views and broadcasting that’s the issue, I think. I’ll compile 1.5 and check.

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [May 21, 2020, 10:06am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/7 "2020-05-21T10:06:15Z")

</div>

To reply to the other comments:

@baggiepinnen I’l be sure to check `@avx` out - thank you!

@ettersi It is indeed a MWE - the real code is a inner calibration loop for the Murchison Widefield Array (a low frequency radio telescope in Australia).

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [May 21, 2020, 10:16am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/8 "2020-05-21T10:16:26Z")

</div>

Julia v1.4.1:

```julia
julia> @btime func1($arr1, $arr2) evals=1 samples=10 seconds=100;
  738.942 ms (76800000 allocations: 3.43 GiB)

julia> @btime func2($arr1, $arr2) evals=1 samples=10 seconds=100;
  68.627 ms (0 allocations: 0 bytes)

```

master (of a couple of weeks ago):

```julia
julia> @btime func1($arr1, $arr2) evals=1 samples=10 seconds=100;
  195.516 ms (0 allocations: 0 bytes)

julia> @btime func2($arr1, $arr2) evals=1 samples=10 seconds=100;
  66.396 ms (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)\
**Post date:** [May 21, 2020, 10:18am UTC](https://discourse.julialang.org/t/why-is-broadcasting-allocating-so-much-memory/39874/9 "2020-05-21T10:18:49Z")

</div>

@giordano That’s great to see they’ve magicked away the allocations - but now I wonder why the slowdown between the two versions on 1.5-master?