# ReverseDiff.GradientTape fails for backslash operator when symmetric sparse matrix involved?

**URL:** <https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318>\
**Category:** New to Julia\
**Tags:** question\
**Created:** [July 19, 2020, 2:56am UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318 "2020-07-19T02:56:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pei](https://avatars.discourse-cdn.com/v4/letter/p/a698b9/32.png) [@Pei](https://discourse.julialang.org/u/Pei)\
**Post date:** [July 19, 2020, 2:56am UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318/1 "2020-07-19T02:56:56Z")

</div>

Hi!  
I am using ReverseDiff to calculate the gradient of a scalar function with respect to its vector input.  
A simplified piece of code is attached in the end. It’s observed the ReverseDiff.GradientTape(loss\_rd, x) breaks when K in the example in symmetric with an error message (truncated) saying “`ERROR: LoadError: MethodError: no method matching lu!(::SparseMatrixCSC{ReverseDiff.TrackedReal{Float64,Float64,Nothing},Int64}, ::Val{true}; check=true)`”, but it works when K is nonsymmetric. I am not sure if I did something wrong in the code. Could someone help me with this? Thanks.

```julia
using ReverseDiff, SparseArrays
function loss_rd(x::AbstractArray{T,1}) where{T}
    f = ones(T, 3)
    K = sparse(I, J, x)
    u = K \ f
    return sum(abs2, u)
end

I = [1, 1, 2, 3, 3] # case that does not work
J = [1, 3, 2, 3, 1]
x = [3.0, 1.0, 3.0, 3.0, 1.0]

#I = [1, 1, 2, 3] # case that works fine
#J = [1, 3, 2, 3]
#x = [3.0, 1.0, 3.0, 3.0]
#
println("loss_rd=", loss_rd(x))

const f_tape = ReverseDiff.GradientTape(loss_rd, x) ###get stuck at this line
const compiled_f_tape = ReverseDiff.compile(f_tape)
dldx=similar(x)
ReverseDiff.gradient!(dldx, compiled_f_tape, x)
println("dl/dx=",dldx)

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 19, 2020, 3:45am UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318/2 "2020-07-19T03:45:23Z")

</div>

ReverseDiff doesn’t support many array/matrix types which is the issue here. Zygote might handle this case better.

---

<div class="post-metadata">

**Author:** ![Pei](https://avatars.discourse-cdn.com/v4/letter/p/a698b9/32.png) [@Pei](https://discourse.julialang.org/u/Pei)\
**Post date:** [July 23, 2020, 7:48pm UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318/3 "2020-07-23T19:48:39Z")

</div>

Thank you for your help, @ChrisRackauckas. I actually started with Zygote and then saw the mutation issue in the original code.  
I tried Zygote for the simplified code here and got the error message, `ERROR: LoadError: Need an adjoint for constructor SparseMatrixCSC{Float64,Int64}. Gradient is of type Array{Float64,2}`. I guess I need to define an adjoint for `sparse(I,J,x)`? After reading the custom adjoint session in Zygote, I get the general idea about how to do this for scalar functions but not for sparse matrix. Could someone point me to some references? Thanks.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 24, 2020, 4:11am UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318/4 "2020-07-24T04:11:09Z")

</div>

Open an issue on Zygote. Looks like it just needs a constructor adjoint. You define those like [Custom Adjoints · Zygote](https://fluxml.ai/Zygote.jl/dev/adjoints/#Custom-Types-1)

---

<div class="post-metadata">

**Author:** ![Pei](https://avatars.discourse-cdn.com/v4/letter/p/a698b9/32.png) [@Pei](https://discourse.julialang.org/u/Pei)\
**Post date:** [July 24, 2020, 2:44pm UTC](https://discourse.julialang.org/t/reversediff-gradienttape-fails-for-backslash-operator-when-symmetric-sparse-matrix-involved/43318/5 "2020-07-24T14:44:02Z")

</div>

Thanks, Chris! I just opened an issue, [https://github.com/FluxML/Zygote.jl/issues/742](https://github.com/FluxML/Zygote.jl/issues/742).