# Does svd() not support forwarddiff?

**URL:** <https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662>\
**Category:** General Usage\
**Tags:** question\
**Created:** [July 23, 2022, 5:16am UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662 "2022-07-23T05:16:44Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [July 23, 2022, 5:16am UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/1 "2022-07-23T05:16:44Z")

</div>

```julia
julia> using LinearAlgebra

julia> using ForwardDiff

julia> function f(x)
       a=[1 2;3 4]
       a=a.*x
       _,s,_=svd(a)
       return sum(s)
       end
f (generic function with 1 method)

julia> ForwardDiff.derivative(f,2)
ERROR: MethodError: no method matching svd!(::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Int64}, Float64, 1}}; full=false, alg=LinearAlgebra.DivideAndConquer())
Closest candidates are:
  svd!(::LinearAlgebra.AbstractTriangular; kwargs...) at ~/julia-1.7.2/share/julia/stdlib/v1.7/LinearAlgebra/src/triangular.jl:2565
  svd!(::StridedMatrix{T}; full, alg) where T<:Union{Float32, Float64, ComplexF32, ComplexF64} at ~/julia-1.7.2/share/julia/stdlib/v1.7/LinearAlgebra/src/svd.jl:97
  svd!(::StridedVector{T}; full, alg) where T<:Union{Float32, Float64, ComplexF32, ComplexF64} at ~/julia-1.7.2/share/julia/stdlib/v1.7/LinearAlgebra/src/svd.jl:106
  ...
Stacktrace:
 [1] svd(A::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Int64}, Int64, 1}}; full::Bool, alg::LinearAlgebra.DivideAndConquer)
   @ LinearAlgebra ~/julia-1.7.2/share/julia/stdlib/v1.7/LinearAlgebra/src/svd.jl:176
 [2] svd(A::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Int64}, Int64, 1}})
   @ LinearAlgebra ~/julia-1.7.2/share/julia/stdlib/v1.7/LinearAlgebra/src/svd.jl:176
 [3] f(x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(f), Int64}, Int64, 1})
   @ Main ./REPL[3]:4
 [4] derivative(f::typeof(f), x::Int64)
   @ ForwardDiff ~/.julia/packages/ForwardDiff/wAaVJ/src/derivative.jl:14
 [5] top-level scope
   @ REPL[4]:1

```

I want to use Forwarddiff.jl to take the derivative of my function, but it seems that it does not support svd. How can I solve this problem? I don’t want to use Zygote.gradient(), which consumes more time and space memory than forward differential

---

<div class="post-metadata">

**Author:** ![DanielVandH](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielvandh/32/31134_2.png) [@DanielVandH](https://discourse.julialang.org/u/DanielVandH)\
**Post date:** [July 23, 2022, 8:26am UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/2 "2022-07-23T08:26:23Z")

</div>

Maybe this is related [Compatibility with Base linear algebra functions · Issue #111 · JuliaDiff/ForwardDiff.jl · GitHub](https://github.com/JuliaDiff/ForwardDiff.jl/issues/111) for why it doesn’t seem to be supported.

In terms of other solutions, maybe [FiniteDiff.jl](https://github.com/JuliaDiff/FiniteDiff.jl)’s `finite_difference_gradient!`/`finite_difference_derivative` meets your needs.

---

<div class="post-metadata">

**Author:** ![maxkapur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maxkapur/32/21208_2.png) [@maxkapur](https://discourse.julialang.org/u/maxkapur)\
**Post date:** [July 23, 2022, 1:30pm UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/3 "2022-07-23T13:30:12Z")

</div>

The discussion @DanielVandH linked mentions [GenericLinearAlgebra.jl](https://julialinearalgebra.github.io/GenericLinearAlgebra.jl/stable/), a project to implement linear algebra functions in pure Julia with support for generic arguments. The `svd` function from this package seems to work just fine with ForwardDiff:

```julia
using GenericLinearAlgebra: svd
using ForwardDiff

function f(x)
    a = [1 2; 3 4]
    a = a .* x
    _, s, _ = svd(a)
    return sum(s)
end

ForwardDiff.derivative(f,2)
# 5.830951894845303

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 23, 2022, 2:32pm UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/4 "2022-07-23T14:32:48Z")

</div>

@mohamed82008 if we make [ImplicitDifferentiation.jl](https://github.com/gdalle/ImplicitDifferentiation.jl) compatible with ForwardDiff.jl, how much work would it be to do the same with [DifferentiableFactorizations.jl](https://github.com/mohamed82008/DifferentiableFactorizations.jl)?

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 23, 2022, 2:37pm UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/5 "2022-07-23T14:37:05Z")

</div>

It would be automatically compatible with FD if ImplicitDifferentiation is.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 23, 2022, 3:45pm UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/6 "2022-07-23T15:45:13Z")

</div>

OK so all we need is to specify the calling behavior of `ImplicitFunction` on `x::AbstractArray{<:Dual}`?  
That would make ForwardDiff.jl a dependency though

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 23, 2022, 3:51pm UTC](https://discourse.julialang.org/t/does-svd-not-support-forwarddiff/84662/7 "2022-07-23T15:51:43Z")

</div>

Yes