I am trying to use ReverseDiff.jl and ForwardDiff.jl as a mixed mode AD approach to computing a Hessian vector product. I am using the following function: import ForwardDiff as fdiff import ReverseDiff as rdiff function _hvp(f::F, x::S, v::S) where {F, S<:AbstractVector{<:AbstractFloat}} dual = f…

Related thread: [image] Mixed-mode automatic differentiation using ForwardDiff and ReverseDiff General Usage I need to take the gradient of a function. I can use ForwardDiff without any issues, but for part of my code I have found ReverseDiff to run much faster. The issue …

The code you posted works for me (assuming Dual, gradient, and partials all come from ForwardDiff.jl): julia> using ForwardDiff: Dual, gradient, partials julia> function _hvp(f::F, x::S, v::S) where {F, S<:AbstractVector{<:AbstractFloat}} dual = Dual.(x, v) return…

Thanks, in my reading of this thread it only seems related in that both the ForwardDiff.jl and ReverseDiff.jl packages are being used, but there is nothing dealing with the composition of them. I think there is perhaps an ambiguity in the term “mixed mode”. In the referenced thread, the poster just …

Ah, yes, apologies for the ambiguity. I updated the question to clarify, but I want to use ReverseDiff.gradient. The goal is to have a forward-over-back approach, where ReverseDiff.jl is handling the backward mode. If I were to use Zygote.gradient instead, then the posted code would work.

Looks like a bug in ReverseDiff, even for the first derivative of this f: julia> ForwardDiff.gradient(f, x) 2-element Vector{Float64}: 0.7876859520316463 1.5891432962932512 julia> Zygote.gradient(f, x) ([0.7876859520316463, 1.589143296293251],) julia> ReverseDiff.gradient(f, x) # wrong 2-eleme…

I am on 1.6.4 and seeing that same issue.

There is this old issue on ReverseDiff.jl that may be related, I am uncertain.

I also see it on the official linux builds for v1.7.1. But I get the correct answers for f2(x) = dot(x, A*x) and f3(x) = dot(x, A, x). Spooky.

Correct Way to Compose ReverseDiff and ForwardDiff

Specific Domains Optimization (Mathematical)

jd-foster July 4, 2022, 2:54am 2

Related thread:

Topic		Replies	Views
Nesting ForwardDiff inside ReverseDiff? General Usage question , differentiation	5	1351	January 4, 2018
Mixed-mode automatic differentiation using ForwardDiff and ReverseDiff General Usage forwarddiff , reversediff , autodiff	9	3000	February 1, 2022
ReverseDiff method error in multiplication General Usage reversediff , autodiff	1	312	December 21, 2022
ReverseDiff.jl Community package , announcement	9	1811	December 13, 2017
Gradiens of gradients using ReverseDiff Numerics	0	652	June 13, 2018

Correct Way to Compose ReverseDiff and ForwardDiff

Related topics