# Do any of the auto differentiation packages support specifying known derivatives by hand?

**URL:** <https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383>\
**Category:** General Usage\
**Created:** [September 23, 2018, 2:38pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383 "2018-09-23T14:38:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [September 23, 2018, 2:38pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/1 "2018-09-23T14:38:20Z")

</div>

Some of the functions I’m trying to do AD over involve integrals, and rather than let the AD fight through my quadrature functions, I want to just use the fundamental theorem of calculus to tell the AD that, essentially,

```julia
d/dx quadgk(f, xmin, x) = f

```

as this shows up in several places throughout the derivation. Is this type of thing supported in any of the AD packages?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [September 23, 2018, 2:47pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/2 "2018-09-23T14:47:36Z")

</div>

Sure, most, if not all, reverse mode AD tools allows this. In Flux its done with a macro @grad if I’m not mistaken.  
[http://fluxml.ai/Flux.jl/stable/internals/tracker.html#Custom-Gradients-1](http://fluxml.ai/Flux.jl/stable/internals/tracker.html#Custom-Gradients-1)

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [September 23, 2018, 4:00pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/3 "2018-09-23T16:00:18Z")

</div>

Wow, that was awesomely easy. I think I did it right, although I’m only just using Flux for the first time:

```julia
using QuadGK
using Flux
using Flux.Tracker: TrackedReal, @grad, data, track, gradient

myquad(f, a, b) = quadgk(f,a,b)[1]
myquad(f, a::TrackedReal, b::TrackedReal) = track(myquad, f, a, b)
myquad(f, a , b::TrackedReal) = track(myquad, f, a, b)
myquad(f, a::TrackedReal, b ) = track(myquad, f, a, b)
@grad myquad(f, a, b) = myquad(f, data(a), data(b)), Δ -> (nothing, -Δ*f(a), Δ*f(b))

f(x) = 2*myquad(x->x^2,x,0)
f′(x) = gradient(f, x)[1]
f′′(x) = gradient(f′, x)[1]

f(1) # -0.6666666666666666
f′(1.) # -2.0 (tracked)
f′′(1.) # -4.0 (tracked)

```

I wonder whether type stability is possible, but I should probably just go read the Flux docs…

I am curious if the other tools have something like this, I was not able to find it in ForwardDiff.jl/ReverseDiff.jl.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 23, 2018, 4:04pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/4 "2018-09-23T16:04:13Z")

</div>

I started something like this in [https://github.com/JuliaDiff/ForwardDiff.jl/pull/165](https://github.com/JuliaDiff/ForwardDiff.jl/pull/165) but it got a bit stale.

---

<div class="post-metadata">

**Author:** ![improbable22](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/improbable22/32/5464_2.png) [@improbable22](https://discourse.julialang.org/u/improbable22)\
**Post date:** [September 23, 2018, 4:48pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/5 "2018-09-23T16:48:47Z")

</div>

This should be dead easy with ForwardDiff too. You essentially need to do this:

```julia
f(x) = x^3

using DualNumbers
f(x::Dual) = dual(f(realpart(x)), df(dualpart(x)))
df(x) = 3x^2

f(dual(1,1))

```

but IIRC it’s just a little more tricky as `ForwardDiff` allows for propagating multiple ɛs at once.

With Autograd it’s a macro a bit like Flux’s `@grad`:

```julia
@primitive f(x),dy,y dy .* fgrad(value(x1)) 

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 23, 2018, 4:52pm UTC](https://discourse.julialang.org/t/do-any-of-the-auto-differentiation-packages-support-specifying-known-derivatives-by-hand/15383/6 "2018-09-23T16:52:35Z")

</div>

It is indeed quite easy for derivatives, but for Jacobians and gradients, you need to do a bit more work (to propagate the partials correctly).