Mutating arrays not supported toy problem

New to Julia
1

Hi all so taking the sample code from the flux gradient documentation I can take the gradient however if I switch out the given function predict(x) = W*x .+ b(the commented line) with a dense layer predict(x) = Dense(5,2)(x) it throws a mutating arrays not supported error. Why is this? I haven’t assigned any values to anything simply changed the predict function.

using Flux

W = rand(2, 5)
b = rand(2)

predict(x) = Dense(5,2)(x)
#predict(x) = W*x .+ b

function loss(x, y)
ŷ = predict(x)
sum((y .- ŷ).^2)
end

x, y = rand(5), rand(2) # Dummy data
loss(x, y) # ~ 3

gs = gradient(() → loss(x, y), params(predict(x)))

Error:

Mutating arrays is not supported – called setindex!(::Vector{Float32}, _…)

Stacktrace:
[1] error(s::String)
@ Base .\error.jl:33
[2] (::Zygote.var"#442#443"{Vector{Float32}})(#unused#::Nothing)
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\lib\array.jl:71
[3] (::Zygote.var"#2339#back#444"{Zygote.var"#442#443"{Vector{Float32}}})(Δ::Nothing)
@ Zygote C:\Users\cfcko.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:67
[4] Pullback
@ .\array.jl:353 [inlined]
[5] (::typeof(∂(fill!)))(Δ::Vector{Float32})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[6] Pullback
@ C:\Users\cfcko.julia\packages\Flux\qAdFM\src\utils.jl:385 [inlined]
[7] (::typeof(∂(create_bias)))(Δ::Vector{Float32})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[8] Pullback
@ C:\Users\cfcko.julia\packages\Flux\qAdFM\src\layers\basic.jl:128 [inlined]
[9] (::typeof(∂(Dense)))(Δ::NamedTuple{(:weight, :bias, :σ), Tuple{Matrix{Float64}, Vector{Float32}, Nothing}})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[10] Pullback
@ C:\Users\cfcko.julia\packages\Flux\qAdFM\src\layers\basic.jl:151 [inlined]
[11] (::typeof(∂(#Dense#160)))(Δ::NamedTuple{(:weight, :bias, :σ), Tuple{Matrix{Float64}, Vector{Float32}, Nothing}})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[12] Pullback (repeats 2 times)
@ C:\Users\cfcko.julia\packages\Flux\qAdFM\src\layers\basic.jl:137 [inlined]
[13] (::typeof(∂(Dense)))(Δ::NamedTuple{(:weight, :bias, :σ), Tuple{Matrix{Float64}, Vector{Float32}, Nothing}})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[14] Pullback
@ .\In[30]:4 [inlined]
[15] (::typeof(∂(predict)))(Δ::Vector{Float64})
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[16] Pullback
@ .\In[30]:8 [inlined]
[17] Pullback
@ .\In[30]:16 [inlined]
[18] (::typeof(∂(#19)))(Δ::Float64)
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface2.jl:0
[19] (::Zygote.var"#94#95"{Zygote.Params, typeof(∂(#19)), Zygote.Context})(Δ::Float64)
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface.jl:357
[20] gradient(f::Function, args::Zygote.Params)
@ Zygote C:\Users\cfcko.julia\packages\Zygote\3I4nT\src\compiler\interface.jl:76
[21] top-level scope
@ In[30]:16
[22] eval
@ .\boot.jl:373 [inlined]
[23] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
@ Base .\loading.jl:1196

2

Hi Christopher!

I’m not sure what you want to achieve with this new prediction function, but there is a slight problem. Not only does predict ignore the values of W and b, it actually creates a new (randomly initialized) dense layer at each call. Which means you cannot learn the parameters because they keep changing :scream:

I suspect this is also the root of your mutation problem, because params(predict(x)) keeps being modified during each call to predict.

In Flux, the way to correct this would be to define the model outside of the prediction function. But since layers are callable, the syntax is actually very easy

predict = Dense(5, 2)

Note that it is very different from

predict(x) = Dense(5, 2)(x)

because in the second one, a new Dense(5, 2) object is created for each x.

Does that answer you question?

1 Like
3

For anyone else landing here with a mutation problem that cannot be resolved in this way, see https://github.com/rakeshvar/Zygote-Mutating-Arrays-WorkAround.jl

1 Like
4

Aaah gotcha perfect thank you. I didn’t want to do anything specific with this function it just happened that I was getting this error in my first attempt at using flux in different code. So I looked up the gradient documentation and they have this snippet as an example of taking a gradient so I was just trying to get it to run with a flux layer instead of the function so I could debug my other code.

5

Glad this solved your problem!

6

@Christopher_Koh can you maybe indicate that my answer was the solution, so that other people visiting this issue know it is closed?

1 Like