# Speed of broadcast operations

**URL:** <https://discourse.julialang.org/t/speed-of-broadcast-operations/52113>\
**Category:** Performance\
**Created:** [December 20, 2020, 4:42am UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113 "2020-12-20T04:42:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![rakeshvar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rakeshvar/32/3613_2.png) [@rakeshvar](https://discourse.julialang.org/u/rakeshvar)\
**Post date:** [December 20, 2020, 4:42am UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113/1 "2020-12-20T04:42:52Z")

</div>

🙇‍♂️

```julia
       M, N = 10000, 10000
       a = randn((M, N));
       ssa = sum(a);
       sa=sum(a, dims=1);
       @btime a .- ssa .- sa;
       @btime a .- sa .- ssa; 
       @btime a .- (sa .+ ssa);
  287.707 ms (6 allocations: 762.94 MiB)
  283.096 ms (6 allocations: 762.94 MiB)
  279.157 ms (6 allocations: 762.94 MiB)

```

Why are all the three operations above running almost in the same time? I would have expected the last one to run in about half the time as the other two; given it has only `1e8` subtractions (and `1e4` additions) as opposed to the first two with `2e8` subtractions.

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [December 20, 2020, 4:54am UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113/2 "2020-12-20T04:54:20Z")

</div>

Your operations (elementwise addition/subtraction) are very cheap, so your runtime will be dominated by the memory traffic needed to fetch each element of `a` from main memory. For reference, on my machine,

```julia
julia> @btime $a .- $ssa .- $sa;
  168.593 ms (2 allocations: 762.94 MiB)

julia> @btime [identity(ai) for ai in a];
  132.609 ms (3 allocations: 762.94 MiB)

```

… it takes almost as much time to do _nothing_ as to do all those operations when the array is so large and the operations are so cheap. The difference probably just amounts to register-shuffling overhead.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [December 20, 2020, 12:59pm UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113/3 "2020-12-20T12:59:41Z")

</div>

You can check how many operations are performed by explicitly counting:

```julia
julia> cnt(x) = (global CNT += 1; x);

julia> CNT=0; cnt.(ones(3) .+ ones(5)'); CNT
15

julia> CNT=0; cnt.(ones(3)) .+ ones(5)'; CNT # fused
15

julia> CNT=0; begin
         col = cnt.(ones(3)) # not fused
         col .+ ones(5)'
       end; CNT
3

```

Maybe you should think of this fused broadcast as a function `(x,y) -> cnt(x) + y` which is mapped over the whole iteration space. For a cheap function that’s a great idea, for an expensive one it isn’t always, and allocating an intermediate like `col` may be worthwhile:

```julia
julia> z = similar(a); # with a, sa as above

julia> @btime $z .= exp.($a) .+ $sa; # N^2 exp calls
  696.363 ms (0 allocations: 0 bytes)

julia> @btime $z .= $a .+ exp.($sa); # fused, still N^2 exp calls
  698.302 ms (0 allocations: 0 bytes)

julia> @btime $z .= $a .+ begin exp.($sa) end; # not fused, N exp calls
  127.645 ms (2 allocations: 78.20 KiB)

```

---

<div class="post-metadata">

**Author:** ![rakeshvar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rakeshvar/32/3613_2.png) [@rakeshvar](https://discourse.julialang.org/u/rakeshvar)\
**Post date:** [December 22, 2020, 3:26am UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113/4 "2020-12-22T03:26:04Z")

</div>

Wow this is really surprising. Is it documented somewhere, the idea of ‘fusing’?  
So would you recommend using intermediate variables while broadcasting?  
I still do not understand my original numbers with simple subtractions! ☹

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 22, 2020, 5:36am UTC](https://discourse.julialang.org/t/speed-of-broadcast-operations/52113/5 "2020-12-22T05:36:02Z")

</div>

[More Dots: Syntactic Loop Fusion in Julia](https://julialang.org/blog/2017/01/moredots/) is the best explanation of why fusing is awesome.