Debugging Zygote: ERROR: Mutating arrays is not supported

New to Julia
,
1

Hi,

I’m fairly new to Julia and not really experienced with Zygote so far, thats why I have a hard time debugging it. I try to implement the following loss function, where I pass a 2 dimensional grid (t and x) and calculate the derrivates of the model for both x and t.
This is my training call:

train!(loss, parameters, [(xtrain, x_grid, t_grid, ytrain)], opt, cb=evalcb)

And here is the definition of my loss function

function loss(xtrain, x_sensor, t_sensor, ytrain)
    f = model(xtrain, vcat(x_sensor, t_sensor))
    f_x = jacobian(x_grid -> model(xtrain, vcat(x_grid, t_sensor)), x_sensor)[1]
    f_t = jacobian(t_grid -> model(xtrain, vcat(x_sensor, t_grid)), t_sensor)[1]

    loss_residual = f_t + f .* f_x
   return sum(loss_residual)

I gett the following error message all the time and I cant figure out what is wrong:

ERROR: Mutating arrays is not supported -- called copyto!(::SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}, _...)
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:33
  [2] (::Zygote.var"#441#442"{SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}})(#unused#::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\array.jl:74
  [3] (::Zygote.var"#2347#back#443"{Zygote.var"#441#442"{SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}})(Δ::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:67
  [4] Pullback
    @ C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\grad.jl:183 [inlined]
  [5] (::typeof(∂(_gradcopy!)))(Δ::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
  [6] Pullback
    @ C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\grad.jl:165 [inlined]
  [7] (::typeof(∂(withjacobian)))(Δ::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
  [8] (::Zygote.var"#208#209"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, typeof(∂(withjacobian))})(Δ::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\lib.jl:207
  [9] #1750#back
    @ C:\Users\Tobi\.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:67 [inlined]
 [10] Pullback
    @ C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\grad.jl:140 [inlined]
 [11] (::typeof(∂(jacobian)))(Δ::Nothing)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [12] Pullback
    @ .\REPL[414]:7 [inlined]
 [13] (::typeof(∂(loss_refactored)))(Δ::Int64)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [14] #208
    @ C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\lib\lib.jl:207 [inlined]
 [15] #1750#back
    @ C:\Users\Tobi\.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:67 [inlined]
 [16] Pullback
    @ C:\Users\Tobi\.julia\packages\Flux\js6mP\src\optimise\train.jl:120 [inlined]
 [17] (::typeof(∂(λ)))(Δ::Int64)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [18] (::Zygote.var"#89#90"{Params{Zygote.Buffer{Any, Vector{Any}}}, typeof(∂(λ)), Zygote.Context})(Δ::Int64)
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface.jl:357
 [19] gradient(f::Function, args::Params{Zygote.Buffer{Any, Vector{Any}}})
    @ Zygote C:\Users\Tobi\.julia\packages\Zygote\DkIUK\src\compiler\interface.jl:76
 [20] macro expansion
    @ C:\Users\Tobi\.julia\packages\Flux\js6mP\src\optimise\train.jl:119 [inlined]
 [21] macro expansion
    @ C:\Users\Tobi\.julia\packages\ProgressLogging\6KXlp\src\ProgressLogging.jl:328 [inlined]
 [22] train!(loss::Function, ps::Params{Zygote.Buffer{Any, Vector{Any}}}, data::Vector{Tuple{Vector{Float64}, Adjoint{Float64, Vector{Float64}}, Matrix{Float64}, Adjoint{Float64, Vector{Float64}}}}, opt::ADAM; cb::typeof(evalcb))
    @ Flux.Optimise C:\Users\Tobi\.julia\packages\Flux\js6mP\src\optimise\train.jl:117
 [23] top-level scope
    @ REPL[415]:1
 [24] top-level scope
    @ C:\Users\Tobi\.julia\packages\CUDA\qAl31\src\initialization.jl:52De
2

Have you implemented model and Jacobean yourself? You are likely mutating some arrays in those implementations. Use o my non-mutating arrays, or Buffers or judiciously annotate with @ignore_derivatives or A combination of those.

Note that generic and even typical Julia code is more often than not INCOMPATIBLE with Zygote (or other AD packages) and it can be nontrivial to rewrite it in a way so that it becomes ADable… at least that’s my personal experience.

3

Zygote.jacobian happens to be one of those incompatible functions: Jacobian in loss function · Issue #953 · FluxML/Zygote.jl · GitHub