Gradient and directional second derivative with ForwardDiff

So I have a function `f` (minus the log-likelihood `ℓ` retrieved from Turing in my case) and I need two things:

1.) gradient `(∇f)(x)` in some point `x`

2.) and how it changes in a direction `θ`:
`∂ₜ(t -> ∇f(x + t*θ))` in `t = 0`

So I figured out how to compute it in one run

``````    return function (y, x, θ)
x_ = x + Dual{:hSrkahPmmC}(0.0, 1.0)*θ
y .= value.(y_)
y, dot(θ, y_).partials[]
end
``````

but I got warned that this might create `tag-order` problems if `ℓ` uses ForwardDiff down the road - perhaps (I am not sure because the outer `gradient` wraps the inner custom tag. Any pointers for me?

I think you want to use:

``````ForwardDiff.derivative(t -> ForwardDiff.gradient(f, x + t * θ), 0)
``````
1 Like

I also need the gradient, `derivative` will only give me the second derivative. I’d hope using a proper `ForwardDiff.Tag(f, typeof(x))` instead of a random symbol may solve it.

Use the DiffResults API of ForwardDiff to get the gradient and its derivative simultaneously. Differentiation API · ForwardDiff!

2 Likes