Correct Way to Compose ReverseDiff and ForwardDiff

RS-Coop
Sun, 03 Jul 2022 16:35:19 +0000
Optimization (Mathematical)

I am trying to use ReverseDiff.jl and ForwardDiff.jl as a mixed mode AD approach to computing a Hessian vector product. I am using the following function:

import ForwardDiff as fdiff
import ReverseDiff as rdiff

function _hvp(f::F, x::S, v::S) where {F, S<:AbstractVector{<:AbstractFloat}}
	dual = fdiff.Dual.(x,v)

	return fdiff.partials.(rdiff.gradient(f, dual), 1)
end

but this is not giving me the expected output. See below for a simple example.

A = randn(2,2)

f(x) = x'*A*x

x = randn(2)
v = randn(2)

_hvp(f,x,v) ≈ (A+A')*v #returns false

What is the correct way to compose these two packages?

9 posts - 5 participants