# Unexpected broadcasting behavior involving eachrow()

**URL:** <https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781>\
**Category:** General Usage\
**Created:** [March 29, 2023, 1:28pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781 "2023-03-29T13:28:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![tomtom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomtom/32/5106_2.png) [@tomtom](https://discourse.julialang.org/u/tomtom)\
**Post date:** [March 29, 2023, 1:28pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/1 "2023-03-29T13:28:26Z")

</div>

in the following, the result of `y2` is counter-intuitive:

```julia
Random.seed!(1);
x = randn(6, 4);

scale = (1 ./ LinearAlgebra.norm.(eachrow(x)) );
y1 = x .* scale;

y2 = copy(x);
y2 .*= (1 ./ LinearAlgebra.norm.(eachrow(y2)) ); # !!! NOT working as expected !!!

julia> LinearAlgebra.norm.(eachrow(y1))
6-element Vector{Float64}:
 1.0
 1.0
 0.9999999999999998
 1.0
 1.0
 1.0

julia> LinearAlgebra.norm.(eachrow(y2))
6-element Vector{Float64}:
 1.009683702121516
 0.9911431758342742
 1.3007113737845422
 1.1961701627839139
 1.1024677374682847
 1.1995854569352082

```

seems like the broadcasting has something strange to do with `eachrow()`.

Actually a lot of _ **parentheses** _ are put to indicate the norm should be calculated _ **before** _ the multiplication but seems like the parentheses are ignored.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 29, 2023, 2:56pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/2 "2023-03-29T14:56:44Z")

</div>

this seems to prevent the problem

```julia
y2 .*= 1 ./ collect(LinearAlgebra.norm.(eachrow(y2)))

```

maybe the `norm()` function is lazy type too!?

---

<div class="post-metadata">

**Author:** ![tomtom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomtom/32/5106_2.png) [@tomtom](https://discourse.julialang.org/u/tomtom)\
**Post date:** [March 29, 2023, 3:14pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/3 "2023-03-29T15:14:37Z")

</div>

it is _ **not** _ good! Why we _ **need** _ to use `collect()` or work out `scale` first?

The parentheses _ **should** _ has given clear enough indication of the order of computation.

I think this behavior causes confusions and bugs.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 29, 2023, 3:40pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/4 "2023-03-29T15:40:21Z")

</div>

Note that broadcasting expands to loop. Therefore norm is repeatedly called for each element of the array (not once per row), because `y2` is two dimensional. Since `y2` is updated in place you do not get what you expect. To see this consider:

```julia
julia> y2 = randn(2, 2)
2×2 Matrix{Float64}:
 -0.753134 -0.149061
 -0.597888 -0.0740529

julia> f(x) = (println(x); norm(x))
f (generic function with 1 method)

julia> y2 .*= (1 ./ f.(eachrow(y2)))
[-0.7531338971077326, -0.14906101769437347]
[-0.5978883610219364, -0.07405291834272695]
[-0.9809709136664748, -0.14906101769437347]
[-0.9924168057314952, -0.07405291834272695]
2×2 Matrix{Float64}:
 -0.980971 -0.150228
 -0.992417 -0.0744119

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 29, 2023, 4:11pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/5 "2023-03-29T16:11:23Z")

</div>

So what’s essentially going on is loop fusion? Where the `.*=` means everything broadcasts to the size of `y2`, rather than the 1-dimension of `eachrow(y2)`? I’m not seeing it

```julia
julia> begin
       y2 = copy(x)
       for j in 1:size(y2, 2)
           for i = 1:size(y2, 1)
               row = y2[i, :]
               n = LinearAlgebra.norm(row)
               inv_n = 1 / n
               y2[i, :] .= y2[i, :] * inv_n
           end
       end
       end

julia> LinearAlgebra.norm.(eachrow(y2))
6-element Vector{Float64}:
 1.0
 1.0
 1.0
 1.0000000000000002
 1.0
 1.0000000000000002

```

---

<div class="post-metadata">

**Author:** ![tomtom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomtom/32/5106_2.png) [@tomtom](https://discourse.julialang.org/u/tomtom)\
**Post date:** [March 29, 2023, 4:20pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/6 "2023-03-29T16:20:17Z")

</div>

I mean its the confusion caused by the trust of using parentheses…

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 29, 2023, 4:30pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/7 "2023-03-29T16:30:50Z")

</div>

No, that’s not the cause. I’ve figured our what the loop is doing. You can see the way it expands.

```julia
julia> begin
       y2 = copy(x)
       y2 .*= 1 ./ LinearAlgebra.norm.(eachrow(y2))
       LinearAlgebra.norm.(eachrow(y2))
       end
6-element Vector{Float64}:
 0.8621336125903504
 1.2096359092602622
 1.0905033928976264
 1.155913402801104
 1.0225581006589461
 1.1695084569556826

julia> begin
       y2 = copy(x)
       for i = 1:size(y2, 1)
           for j in 1:size(y2, 2)
               row = y2[i, :]
               n = LinearAlgebra.norm(row)
               inv_n = 1 / n
               y2[i, j] = y2[i, j] * inv_n
           end
       end
       LinearAlgebra.norm.(eachrow(y2))
       end
6-element Vector{Float64}:
 0.8621336125903504
 1.2096359092602622
 1.0905033928976264
 1.155913402801104
 1.0225581006589461
 1.1695084569556826

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 29, 2023, 4:40pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/8 "2023-03-29T16:40:05Z")

</div>

These examples seem to work as expected.  
But I don’t understand how they differ (structurally) from the original example.

```julia
julia> using StatsBase

julia> N=copy(M)
3×2 Matrix{Float64}:
 2.0 1.0
 3.0 1.0
 4.0 1.0
julia> N = N .* mean.(eachrow(N))
3×2 Matrix{Float64}:
  3.0 1.5
  6.0 2.0
 10.0 2.5

julia> N=copy(M)
3×2 Matrix{Float64}:
 2.0 1.0
 3.0 1.0
 4.0 1.0

julia> N = N .* (1 .* mean.(eachrow(N)))
3×2 Matrix{Float64}:
  3.0 1.5
  6.0 2.0
 10.0 2.5

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 29, 2023, 4:40pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/9 "2023-03-29T16:40:42Z")

</div>

> [@pdeffebach](#):
>
> Where the `.*=` means everything broadcasts to the size of `y2`, rather than the 1-dimension of `eachrow(y2)`?

Yes, which means that `norm` is called for each element of a row separately and it sees updated values in `y2`. The problem is that:

```julia
julia> x = rand(2,2)
2×2 Matrix{Float64}:
 0.752214 0.337909
 0.190713 0.866113

julia> Base.mightalias(x, eachrow(x))
false

```

I have opened [Improve handling of aliasing of AbstractSlices by bkamins · Pull Request #49182 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/49182) to fix it.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 29, 2023, 4:41pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/10 "2023-03-29T16:41:11Z")

</div>

> [@rocco\_sprmnt21](#):
>
> I don’t understand how they differ (structurally) from the original example.

They are not in place, so there is no issue with aliasing.

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [March 29, 2023, 5:09pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/11 "2023-03-29T17:09:27Z")

</div>

```julia
julia> y3 = copy(x);

julia> y3 .*= (1 ./ Iterators.map(norm, eachrow(y3)));

julia> norm.(eachrow(y3))
6-element Vector{Float64}:
 1.0
 1.0
 0.9999999999999998
 1.0
 1.0
 1.0

```

works as desired. And since `Iterators.map` is an iterator, it isn’t evaluated O(n\*m) times but evaluated O(n) times and values reused.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [March 29, 2023, 5:25pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/12 "2023-03-29T17:25:59Z")

</div>

> [@bkamins](#):
>
> Yes, which means that `norm` is called for each element of a row separately

besides correctness, this seems like an enormous performance footgun

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [March 29, 2023, 5:34pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/13 "2023-03-29T17:34:54Z")

</div>

This has been discussed before. For _pure functions_, this is a footgun, but when sideeffects of the function are desired, it isn’t. But somebody who particiapated in those previous discussions can weigh in.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [March 29, 2023, 5:40pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/14 "2023-03-29T17:40:40Z")

</div>

```julia
using LinearAlgebra: norm
x = rand(6, 4)
x ./= norm.(eachrow(x))
norm.(eachrow(x)) # returns something very surprising!

```

There’s absolutely no world I want to live in where the above snippet doesn’t normalize the rows’ norms to 1.

If side-effects are desired then surely there is another (more clear, and less magical) implementation that can achieve what the above snippet _currently_ does

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 29, 2023, 9:05pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/15 "2023-03-29T21:05:41Z")

</div>

> [@adienes](#):
>
> There’s absolutely no world I want to live in where the above snippet doesn’t normalize the rows’ norms to 1.

As commented above - this is the issue with dealiasing by broadcasting. So hopefully my PR to Julia would be merged and the issue will be fixed (still - it would be better to precompute `norm` to get better performance)

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [March 29, 2023, 9:22pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/16 "2023-03-29T21:22:04Z")

</div>

> [@Dan](#):
>
> `y3 .*= (1 ./ Iterators.map(norm, eachrow(y3)));`

Broadcasting an iterator collects it first, e.g. `Broadcast.broadcastable(Iterators.map(sqrt, 1:4))`. So this is the same as writing eager `map`.

> [@adienes](#):
>
> ```julia
> x ./= norm.(eachrow(x))
> norm.(eachrow(x)) # returns something very surprising!
> 
> ```
> 
> There’s absolutely no world I want to live in where the above snippet doesn’t normalize the rows’ norms to 1.

It’s worse, as `x ./= norm.(eachcol(x))` also fails for other reasons. You need this mouthful: `x ./= map(norm, eachslice(x; dims=1, drop=false))`.

> [@](#):
>
> If side-effects are desired then surely there is another (more clear, and less magical) implementation that can achieve what the above snippet _currently_ does

For `sum` instead of norm, what works fine is `x ./= sum(x; dims=2)`. IMO we should have `norm(x; dims=2)`, which besides avoiding aliasing problems, can be much faster.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [March 29, 2023, 9:26pm UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/17 "2023-03-29T21:26:50Z")

</div>

Besides @bkamins 's PR (for which I am not an expert reviewer but it seems the right way to go), would it be feasible to emit a compile-time `AliasWarning` or something like that when this kind of construction is encountered?

---

<div class="post-metadata">

**Author:** ![tomtom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomtom/32/5106_2.png) [@tomtom](https://discourse.julialang.org/u/tomtom)\
**Post date:** [March 30, 2023, 12:59am UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/18 "2023-03-30T00:59:05Z")

</div>

yeah. Again, the _ **violation against the intuitive use of parentheses** _ is very dangerous I think.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [March 30, 2023, 1:41am UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/19 "2023-03-30T01:41:37Z")

</div>

This doesn’t seem like an issue with parentheses and order of operations at all, it seems like not recognizing that chained dot syntax fuses the operations into 1 loop. For a simpler example, `a .* (b .+ c)` does `a[i] * (b[i] + c[i])` per iteration. Note that the parentheses does not prevent fusing, in fact it sensibly specifies the order of operations per iteration. The only way to split that example into separate loops is to broadcast separately: `temp = b .+ c; a .* temp`.

Generally, if your input is being changed per iteration, results will be different if loop fusion has a reducing operation like `norm(eachrow(...`; put plainly, reducing operations depend on the other elements, including ones changed by previous iterations. `y1` was computed by broadcasting two times over an input `x`, which did not change in the process. `y2` was changing over its own broadcast. If the iterations’ inputs are different, of course the output is different.

Changing the input to an unchanging `x` is a simple way to fix the issue: `y2 .*= (1 ./ LinearAlgebra.norm.(eachrow(x)) );`. A previous comment suggested `collect`, which split the line into 2 broadcast loops and made unchanging temporary data based on `y2` before it was changed.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [March 30, 2023, 2:19am UTC](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781/20 "2023-03-30T02:19:29Z")

</div>

> [@adienes](#):
>
> ```julia
> x ./= norm.(eachrow(x))
> norm.(eachrow(x)) # returns something very surprising!
> 
> ```
> 
> There’s absolutely no world I want to live in where the above snippet doesn’t normalize the rows’ norms to 1.

Again the problem is that the in-place loop has a reducing operation; the changing output is also the input. In the linked PR you argued that the equivalent NumPy code does not have this issue, but it is allocating temporary arrays in separate loops of each operation. Julia’s fused broadcast loops may look similar but are not equivalent, and I think people should understand the difference.

People have suggested manually inserting unbroadcasted `identity` calls into a single line to unfuse loops and allocate temporary arrays, like `x ./= identity(norm.(eachrow(x)))`. But that’s a lot of bloat with more dots, and a single line could be transformed by a macro like `@unfuse x ./= norm.(eachrow(x))`

[Next page](https://discourse.julialang.org/t/unexpected-broadcasting-behavior-involving-eachrow/96781.md?page=2)