Zygote hessian

I’m trying to use Zygote to compute a simple Hessian. What am I doing wrong?

using Zygote
p = 0
function ϕ(x, p)
    return sum( x.^2 )
end
gϕ = x -> Zygote.gradient(xx->ϕ(xx,p), x)
Hϕ = x -> Zygote.hessian(xx->ϕ(xx,p), x)
x0 = [-1.1, 2.1]
gϕ(x0)
Hϕ(x0)

The gradient works, but for the Hessian, I get an error:

ERROR: Need an adjoint for constructor ForwardDiff.Dual{Nothing,Float64,2}. Gradient is of type ForwardDiff.Dual{Nothing,Float64,2}
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] (::Zygote.Jnew{ForwardDiff.Dual{Nothing,Float64,2},Nothing,false})(::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/lib.jl:255
 [3] (::Zygote.var"#287#back#162"{Zygote.Jnew{ForwardDiff.Dual{Nothing,Float64,2},Nothing,false}})(::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/ZygoteRules/Jn4LO/src/adjoint.jl:48
 [4] Dual at /Users/jakeroth/.julia/packages/ForwardDiff/N0wMF/src/dual.jl:19 [inlined]
 [5] (::typeof(∂(ForwardDiff.Dual{Nothing,Float64,2})))(::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/compiler/interface2.jl:0
 [6] literal_pow at /Users/jakeroth/.julia/packages/ForwardDiff/N0wMF/src/dual.jl:55 [inlined]
 [7] (::Zygote.var"#1241#1248")(::typeof(∂(literal_pow)), ::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/broadcast.jl:113
 [8] #3 at ./generator.jl:36 [inlined]
 [9] iterate at ./generator.jl:47 [inlined]
 [10] collect(::Base.Generator{Base.Iterators.Zip{Tuple{Array{typeof(∂(literal_pow)),1},FillArrays.Fill{ForwardDiff.Dual{Nothing,Float64,2},1,Tuple{Base.OneTo{Int64}}}}},Base.var"#3#4"{Zygote.var"#1241#1248"}}) at ./array.jl:622
 [11] map at ./abstractarray.jl:2155 [inlined]
 [12] (::Zygote.var"#1240#1247"{Tuple{Base.RefValue{typeof(^)},Array{ForwardDiff.Dual{Nothing,Float64,2},1},Base.RefValue{Val{2}}},Val{4},Array{typeof(∂(literal_pow)),1}})(::FillArrays.Fill{ForwardDiff.Dual{Nothing,Float64,2},1,Tuple{Base.OneTo{Int64}}}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/broadcast.jl:113
 [13] #3186#back at /Users/jakeroth/.julia/packages/ZygoteRules/Jn4LO/src/adjoint.jl:48 [inlined]
 [14] #143 at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/lib.jl:132 [inlined]
 [15] #252#back at /Users/jakeroth/.julia/packages/ZygoteRules/Jn4LO/src/adjoint.jl:48 [inlined]
 [16] broadcasted at ./broadcast.jl:1237 [inlined]
 [17] ϕ at ./REPL[68]:2 [inlined]
 [18] (::typeof(∂(ϕ)))(::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/compiler/interface2.jl:0
 [19] #90 at ./REPL[70]:1 [inlined]
 [20] (::Zygote.var"#32#33"{typeof(∂(#90))})(::ForwardDiff.Dual{Nothing,Float64,2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/compiler/interface.jl:38
 [21] gradient(::Function, ::Array{ForwardDiff.Dual{Nothing,Float64,2},1}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/compiler/interface.jl:47
 [22] (::Zygote.var"#1303#1304"{var"#90#92"})(::Array{ForwardDiff.Dual{Nothing,Float64,2},1}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/utils.jl:81
 [23] forward_jacobian(::Zygote.var"#1303#1304"{var"#90#92"}, ::Array{Float64,1}, ::Val{2}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/forward.jl:23
 [24] forward_jacobian(::Function, ::Array{Float64,1}) at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/forward.jl:38
 [25] hessian at /Users/jakeroth/.julia/packages/Zygote/bdE6T/src/lib/utils.jl:81 [inlined]
 [26] (::var"#89#91")(::Array{Float64,1}) at ./REPL[70]:1
 [27] top-level scope at REPL[73]:100:
2 Likes

For what it is worth, this example now runs fine in Zygote.

3 Likes

when I calculate the gradient of a function consiting of LinearAlgebra.diagm, I encounter similar error:
Need an adjoint for constructor Pair{Int64,Array{Float64,1}}
How can I solve this problem?