# Mutating versus non-mutating arrays for Zygote Gradient

**URL:** <https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095>\
**Category:** General Usage\
**Created:** [December 24, 2022, 10:58pm UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095 "2022-12-24T22:58:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![erlebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erlebach/32/12973_2.png) [@erlebach](https://discourse.julialang.org/u/erlebach)\
**Post date:** [December 24, 2022, 10:58pm UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/1 "2022-12-24T22:58:48Z")

</div>

I created a custom `Lux` layer to evaluate a polynomial. Now, I have `N` point entering the layer, transformed, and input into a loss function. I created a loop to evaluate the polynomial since I do not understand broadcasting enough to do it. But I then realized that my loops are mutating and Zygote will not be able to calculate the gradient of such a function. Here is the code:

```julia
function (l::Polylayer)(x::AbstractMatrix, ps, st::NamedTuple)
    c = ps.coeffs
    x1 = reshape(x, l.out_dims, :) # lasts dimension is the number of training samples

    N = size(x1, length(size(x1)))
    sum = zeros(l.out_dims, N)

    for i in 1:N
        sum[:, i] .= c[:, end]
    end
    for d in l.degree : -1 : 1
        for i in 1:N
            sum[:, i] .= sum[:, i] .* x1[:, i] .+ c[:, d]
        end
    end
    return sum, st
end

```

Clearly, the line ` sum[:, i] .= sum[:, i] .* x1[:, i] .+ c[:, d]` is problematic if I want to take a gradient.  
So my question is, how can I transform the loop using broadcast?

Thanks,

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 24, 2022, 11:08pm UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/2 "2022-12-24T23:08:22Z")

</div>

Just use mapping constructs instead.

```julia
function (l::Polylayer)(x::AbstractMatrix, ps, st::NamedTuple)
    c = ps.coeffs
    x1 = reshape(x, l.out_dims, :) # lasts dimension is the number of training samples

    N = size(x1, length(size(x1)))
    sum = zeros(l.out_dims, N)

    sum2 = reduce(hcat,map(1:N) do i
         c[:, end]
    end)
    for d in l.degree : -1 : 1
        sum2 = reduce(hcat,map(1:N) do i
            sum2[:, i] .* x1[:, i] .+ c[:, d]
        end)
    end
    return sum2, st
end

```

---

<div class="post-metadata">

**Author:** ![erlebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erlebach/32/12973_2.png) [@erlebach](https://discourse.julialang.org/u/erlebach)\
**Post date:** [December 24, 2022, 11:12pm UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/3 "2022-12-24T23:12:48Z")

</div>

I have here the source to `evalpoly`. It looks like this is a mutating implementation:

```julia
function evalpoly(x, p::Tuple)
    if @generated
        N = length(p.parameters::Core.SimpleVector)
        ex = :(p[end])
        for i in N-1:-1:1
            ex = :(muladd(x, $ex, p[$i]))
        end
        ex
    else
        _evalpoly(x, p)
    end
end

```

Isn’t `ex` overwritten? Why won’t `Zygote` have a problem with this?  
In my code, I have an array and am mutating its elements, which Zygote does not like.

If I evaluate `evalpoly` with a vector argument and the `.evalpoly` notation (notice the dot), am I not mutating? For example,

```julia
y .= evalpoly(x, Ref((1,0,2))) # Mutating or non-mutating?

```

where `y` is a vector of size 128, for example. Should one replace `.=` by `=` to make it mutating, at the expense of using more memory? Thanks.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 24, 2022, 11:17pm UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/4 "2022-12-24T23:17:03Z")

</div>

> [@erlebach](#):
>
> Isn’t `ex` overwritten? Why won’t `Zygote` have a problem with this?

It’s overwriting, not mutating.

---

<div class="post-metadata">

**Author:** ![erlebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erlebach/32/12973_2.png) [@erlebach](https://discourse.julialang.org/u/erlebach)\
**Post date:** [December 25, 2022, 12:26am UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/5 "2022-12-25T00:26:47Z")

</div>

Got it. I know I can unroll the polynomials of course, but I will want to handle the multivariate case. I wonder if I can use the `ModelingToolkit` to help with this task. I also have to read why Zygote cannot handle mutations, or perhaps it is only the forward differentiation that must be immutable? I have not read much about this as yet.

After reading documentation and considering different solution, I have decided to look into a custom derivative rule with ChainCore.jl. We’ll see how that goes. If it doesn’t, I might have to try using Python, even though it is much slower. I have to take development time into consideration 🙂 . Ughh.

Merry Christmas!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 25, 2022, 2:08am UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/6 "2022-12-25T02:08:57Z")

</div>

> [@erlebach](#):
>
> Isn’t `ex` overwritten?

Besides the fact that assignment is not the same as mutation, I should point out that this `ex` transformation is not runtime code — it’s not what the AD system is analyzing — it’s metaprogramming code that runs at compile time (because this is a `@generated` function).

---

<div class="post-metadata">

**Author:** ![erlebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erlebach/32/12973_2.png) [@erlebach](https://discourse.julialang.org/u/erlebach)\
**Post date:** [December 25, 2022, 2:57am UTC](https://discourse.julialang.org/t/mutating-versus-non-mutating-arrays-for-zygote-gradient/92095/7 "2022-12-25T02:57:09Z")

</div>

Yes. In fact, a mask is probably what I need to solve my problem without mutation. But that takes me Ina direction I don’t have time for at the moment. Thanks for the observation.

Cheers,