ForwardDiff.jl : ERROR: LoadError: MethodError: convert(...) is ambiguous

General Usage
, , ,
1

My original intention was to use Zygote to achieve the gradient of gradients, but there were a lot of problems with twice automatic differentiation in reverse mode. So I used Zygote.hessian to achieve the mixing forward + reverse mode, and then some problems occurred. The following is my example

using GenericLinearAlgebra
using Zygote

function f(x)
    A = reshape(sin.(collect(1:25)*x),5,5)
    u, s,_= GenericLinearAlgebra.svd(A)
    A = u *A *u
    A = A / maximum(abs.(A))
    return sum(A)
end
function test()
    x0=0.5
    @show gradient(f,x0)
    @show hessian(f,x0)

end

test()

It give

gradient(f, x0) = (64.0,)
ERROR: LoadError: MethodError: convert(::Type{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, ::ChainRulesCore.ZeroTangent) is ambiguous. Candidates:
  convert(::Type{T}, x::ChainRulesCore.AbstractZero) where T<:Number in ChainRulesCore at ~\.julia\packages\ChainRulesCore\ctmSK\src\tangent_types\abstract_zero.jl:31
  convert(::Type{ForwardDiff.Dual{T, V, N}}, x) where {T, V, N} in ForwardDiff at ~\.julia\packages\ForwardDiff\pDtsf\src\dual.jl:432
Possible fix, define
  convert(::Type{ForwardDiff.Dual{T, V, N}}, ::ChainRulesCore.AbstractZero) where {T, V, N}
Stacktrace:
  [1] fill!(dest::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, x::ChainRulesCore.ZeroTangent)
    @ Base .\array.jl:351
  [2] _setindex_zero(::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, ::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}, ::Int64, ::Vararg{Int64})
    @ ChainRules ~\.julia\packages\ChainRules\DUopG\src\rulesets\Base\indexing.jl:104
  [3] ∇getindex(x::Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, dy::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}, inds::CartesianIndex{2})
    @ ChainRules ~\.julia\packages\ChainRules\DUopG\src\rulesets\Base\indexing.jl:89
  [4] (::ChainRules.var"#1564#1566"{Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}, Tuple{CartesianIndex{2}}})()
    @ ChainRules ~\.julia\packages\ChainRules\DUopG\src\rulesets\Base\indexing.jl:74
  [5] unthunk
    @ ~\.julia\packages\ChainRulesCore\ctmSK\src\tangent_types\thunks.jl:199 [inlined]
  [6] unthunk(x::ChainRulesCore.InplaceableThunk{ChainRulesCore.Thunk{ChainRules.var"#1564#1566"{Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}, Tuple{CartesianIndex{2}}}}, ChainRules.var"#1563#1565"{Matrix{ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1}, Tuple{CartesianIndex{2}}}})
    @ ChainRulesCore ~\.julia\packages\ChainRulesCore\ctmSK\src\tangent_types\thunks.jl:232
  [7] wrap_chainrules_output
    @ ~\.julia\packages\Zygote\xGkZ5\src\compiler\chainrules.jl:105 [inlined]
  [8] map
    @ .\tuple.jl:222 [inlined]
  [9] wrap_chainrules_output
    @ ~\.julia\packages\Zygote\xGkZ5\src\compiler\chainrules.jl:106 [inlined]
 [10] ZBack
    @ ~\.julia\packages\Zygote\xGkZ5\src\compiler\chainrules.jl:206 [inlined]
 [11] Pullback
    @ ~\Desktop\Ising_test\test.jl:8 [inlined]
 [12] (::typeof(∂(f)))(Δ::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1})
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\compiler\interface2.jl:0
 [13] (::Zygote.var"#60#61"{typeof(∂(f))})(Δ::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1})
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\compiler\interface.jl:45
 [14] gradient(f::Function, args::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1})
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\compiler\interface.jl:97
 [15] (::Zygote.var"#104#105"{typeof(f)})(x::ForwardDiff.Dual{ForwardDiff.Tag{Zygote.var"#104#105"{typeof(f)}, Float64}, Float64, 1})
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\lib\grad.jl:66
 [16] derivative(f::Zygote.var"#104#105"{typeof(f)}, x::Float64)
    @ ForwardDiff ~\.julia\packages\ForwardDiff\pDtsf\src\derivative.jl:14
 [17] hessian_dual(f::Function, x::Float64)
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\lib\grad.jl:66
 [18] hessian(f::Function, x::Float64)
    @ Zygote ~\.julia\packages\Zygote\xGkZ5\src\lib\grad.jl:62
 [19] macro expansion
    @ .\show.jl:1047 [inlined]
 [20] test()
    @ Main ~\Desktop\Ising_test\test.jl:14
 [21] top-level scope
    @ ~\Desktop\Ising_test\test.jl:20
in expression starting at ~\Desktop\Ising_test\test.jl:20

It seems that there is a problem when calling ForwardDiff, but I don’t know how to fix it

2

Playing with this by deleting things from your function until I get something similar that triggers the same error, I found this:

function f3(x)
    A = ones(5,5)*x
    maximum(A)
end

Which gives the same error. I think the most important part of the error is

Possible fix, define
  convert(::Type{ForwardDiff.Dual{T, V, N}}, ::ChainRulesCore.AbstractZero) where {T, V, N}

This definition seems to work, but I am not confident it is correct:

using ForwardDiff, ChainRulesCore

Base.convert(::Type{ForwardDiff.Dual{T, V, N}}, ::ChainRulesCore.AbstractZero) where {T, V, N} = ForwardDiff.Dual{T, V, N}(zero(V))
3

I think that’s a new bug, from these lines:

Can you make an issue on ChainRules?

This looks fine to me, for now. But CR and ForwardDiff don’t depend on each other, so some other solution ought to be invented longer-term.

4

This seems to be correct in most cases, but something very strange happened in my project. I’m sorry I can’t reproduce the same error with an example. It has nan type, which is really weird. I’ve avoided it by writing it in an equivalent way, but I don’t know what the problem is. But that’s okay. I don’t want to dwell on it right now. Because there is a new error, I will start a new topic to describe it.

5

This is a new problem I encountered with forwarddiff. I wonder if you are interested in it