# Gradient and directional second derivative with ForwardDiff

**URL:** <https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429>\
**Category:** Probabilistic Programming\
**Tags:** autodiff\
**Created:** [June 23, 2021, 9:35am UTC](https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429 "2021-06-23T09:35:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [June 23, 2021, 9:35am UTC](https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429/1 "2021-06-23T09:35:50Z")

</div>

So I have a function `f` (minus the log-likelihood `ℓ` retrieved from Turing in [my case](https://github.com/mschauer/ZigZagBoomerang.jl/blob/82399148675f2362cb05e24a32e50eb679ee89b4/turing/lr.jl#L64)) and I need two things:

1.) gradient `(∇f)(x)` in some point `x`

2.) and how it changes in a direction `θ`:  
`∂ₜ(t -> ∇f(x + t*θ))` in `t = 0`

So I figured out how to compute it in _one run_

```julia
    return function (y, x, θ)
        x_ = x + Dual{:hSrkahPmmC}(0.0, 1.0)*θ
        y_ = ForwardDiff.gradient(x->-ℓ(x), x_)
        y .= value.(y_)
        y, dot(θ, y_).partials[]
    end

```

but I got warned that this might create `tag-order` problems if `ℓ` uses ForwardDiff down the road - perhaps (I am not sure because the outer `gradient` wraps the inner custom tag. Any pointers for me?

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [June 23, 2021, 12:47pm UTC](https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429/2 "2021-06-23T12:47:56Z")

</div>

I think you want to use:

```julia
ForwardDiff.derivative(t -> ForwardDiff.gradient(f, x + t * θ), 0)

```

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [June 23, 2021, 12:58pm UTC](https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429/3 "2021-06-23T12:58:07Z")

</div>

> [@mschauer](#):
>
> y, dot(θ, y\_).partials

I also need the gradient, `derivative` will only give me the second derivative. I’d hope using a proper `ForwardDiff.Tag(f, typeof(x))` instead of a random symbol may solve it.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [June 23, 2021, 1:04pm UTC](https://discourse.julialang.org/t/gradient-and-directional-second-derivative-with-forwarddiff/63429/4 "2021-06-23T13:04:39Z")

</div>

Use the DiffResults API of ForwardDiff to get the gradient and its derivative simultaneously. [Differentiation API · ForwardDiff](https://juliadiff.org/ForwardDiff.jl/stable/user/api/#ForwardDiff.derivative)!