# Automatic differentiation for higher-order derivatives

**URL:** <https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337>\
**Category:** General Usage\
**Tags:** forwarddiff\
**Created:** [June 3, 2021, 2:33pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337 "2021-06-03T14:33:12Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [June 3, 2021, 2:33pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/1 "2021-06-03T14:33:12Z")

</div>

Hi,  
I would like to get some results from 0th to nth derivatives evaluation of a given function.  
For example,  
`my_func(x::Real)` returns `(f(x), f'(x), f''(x), ..., f^{n}(x))`.  
Note that [DiffResults.jl](https://github.com/JuliaDiff/DiffResults.jl) with [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl) may be what I want but it seems only give `f(x), f(x)`, and `H(x)`.

Any ideas?

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [June 3, 2021, 4:51pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/2 "2021-06-03T16:51:36Z")

</div>

I do that [here](https://github.com/rveltz/BifurcationKit.jl/blob/master/src/Utils.jl#L194)

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [June 3, 2021, 5:01pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/3 "2021-06-03T17:01:05Z")

</div>

Thanks 🙂  
BTW, is it efficient? I mean, DiffResults.jl claims that the calculation of multiple orders of automatic differentiation would be redundant.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [June 3, 2021, 5:08pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/4 "2021-06-03T17:08:11Z")

</div>

> BTW, is it efficient

In what terms?

I use it for GPU based applications, it works well.

---

<div class="post-metadata">

**Author:** ![Keno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keno/32/285_2.png) [@Keno](https://discourse.julialang.org/u/Keno)\
**Post date:** [June 3, 2021, 5:20pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/5 "2021-06-03T17:20:28Z")

</div>

Higher order AD is a complicated subject, but the current best guidance I would have is to use ForwardDiff for scalar functions up to fourth order or so, and TaylorSeries.jl beyond that. For non-scalar code we don’t currently have a great answer, but Diffractor.jl will provide this functionality once released.

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 3, 2021, 7:45pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/6 "2021-06-03T19:45:54Z")

</div>

May I ask what are your time estimates regarding diffractor? I am currently calculating 2nd order gradients with Zygote and the first gradient takes some time, more than 10 minutes.

---

<div class="post-metadata">

**Author:** ![Keno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keno/32/285_2.png) [@Keno](https://discourse.julialang.org/u/Keno)\
**Post date:** [June 3, 2021, 9:06pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/7 "2021-06-03T21:06:12Z")

</div>

Soon. I’ve sent it to some people for early comments, but it isn’t quite ready yet to try widely and I want to avoid people being disappointed. If your code is public, I’d be happy to give it a try with Diffractor just to see if it works or not (and if not, put it on my list of things to fix).

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [June 3, 2021, 9:47pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/8 "2021-06-03T21:47:27Z")

</div>

```julia
function ndiff(f,x,n)
    tag = ForwardDiff.Dual{nothing}(x,one(x))
    for _ in 1:n
        tag = ForwardDiff.Dual{nothing}(tag,one(tag))
    end
    rawres = f(tag)
    return rawres
end

```

basically `rawres` should have all the derivative information, but im looking on how to extract it. diffresults stores this dual and makes it easier to obtain the proper information

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [June 4, 2021, 4:12am UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/9 "2021-06-04T04:12:31Z")

</div>

Inspired by you, I wrote a code:

```julia
    if auto_diff
        _funcs = Function[x_cmd_func]
        for i in 1:d+1
            push!(_funcs, (t) -> ForwardDiff.derivative(_funcs[i], t))
        end
    end
    funcs(t) = [_func(t) for _func in _funcs]

```

---

<div class="post-metadata">

**Author:** ![lbenet](https://avatars.discourse-cdn.com/v4/letter/l/35a633/32.png) [@lbenet](https://discourse.julialang.org/u/lbenet)\
**Post date:** [June 4, 2021, 1:13pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/10 "2021-06-04T13:13:30Z")

</div>

Hi,

you may also take a look in [TaylorSeries.jl](https://github.com/JuliaDiff/TaylorSeries.jl). The n-th coefficient of the Taylor series expansion of f(x+a) is the derivative f^{(n)}(a)/n!, which I think is what you are looking for.

In code, you do the following:

```julia
julia> using TaylorSeries

julia> t = Taylor1(5) # "independent" variable of order (degree) 5
 1.0 t + 𝒪(t⁶)

julia> sin(t) # Taylor series around 0 of `sin`
 1.0 t - 0.16666666666666666 t³ + 0.008333333333333333 t⁵ + 𝒪(t⁶)

julia> sin(1.0+t) # Taylor series around `1.0 of `sin`
 0.8414709848078965 + 0.5403023058681398 t - 0.42073549240394825 t² - 0.09005038431135663 t³ + 0.03506129103366235 t⁴ + 0.004502519215567832 t⁵ + 𝒪(t⁶)

```

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 4, 2021, 7:07pm UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/11 "2021-06-04T19:07:08Z")

</div>

Hi Keno,

thanks for the offer. Unfortunately, the code is not yet public, as it is part of my research work. I can try to carve out some MWE and send it to you as an example. Would it work for you?

Tomas

---

<div class="post-metadata">

**Author:** ![Keno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/keno/32/285_2.png) [@Keno](https://discourse.julialang.org/u/Keno)\
**Post date:** [June 7, 2021, 12:40am UTC](https://discourse.julialang.org/t/automatic-differentiation-for-higher-order-derivatives/62337/12 "2021-06-07T00:40:01Z")

</div>

Sure, if it’s easy to do. Don’t spend to much time on it though.