# Automatic differentiation of function with special points

**URL:** <https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780>\
**Category:** General Usage\
**Tags:** forwarddiff, autodiff\
**Created:** [January 30, 2023, 4:05pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780 "2023-01-30T16:05:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![rpgowers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rpgowers/32/46229_2.png) [@rpgowers](https://discourse.julialang.org/u/rpgowers)\
**Post date:** [January 30, 2023, 4:05pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780/1 "2023-01-30T16:05:13Z")

</div>

I am using ForwardDiff.jl to compute derivatives of functions which often contain functions of the form:

f(x,y) = x/[exp(x/y)-1]

which one can show using L’Hôpital’s rule has the value f(0,y) = y. One way to construct such a function is:

```julia
f(x, y) = (x == 0.0 ? y : x/(exp(x/y)-1) )

```

This is continuous with respect to x, but ForwardDiff understandably gives an x-derivative of zero at x=0. The actual derivative at x=0 should be f’(0,y) = -1/2. If I only required the first derivative, then I could put in a workaround to take this into account (e.g. `x == 0.0 ? y-x/2 : ...` ). However, I also intend to use the second and third derivatives wrt x and I’d like to know how to generalise to similar functions.

Is there a way to do this by defining custom derivatives? Or put another way, why does ForwardDiff work on the `sinc` function when it also contains a special point similar to this at x=0?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 30, 2023, 5:03pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780/3 "2023-01-30T17:03:10Z")

</div>

ForwardDiff should ignore branches like `x == 0.0` as these often lead to wrong answers (like here). For various reasons this is only on master branch right now. It leads to NaN instead of a wrong answer.

One way around this is to splice in a Taylor expansion on a finite region. A very quick attempt (without thinking carefully about how it will behave at very small nonzero `x`) looks like this:

```julia
julia> f1(x, y) = x/(exp(x/y)-1);
julia> f2(x, y) = x == 0.0 ? y : x/(exp(x/y)-1);
julia> f3(x, y) = abs2(x)>eps() ? x/(exp(x/y)-1) : y - x/2 + x^2/(12*y) - x^4/(720*y^3);

julia> using ForwardDiff # master, i.e. v0.11

julia> ForwardDiff.derivative(x -> f1(x, 1), 0)
NaN

julia> ForwardDiff.derivative(x -> f2(x, 1), 0) # was 0 on v0.10
NaN

julia> ForwardDiff.derivative(x -> f3(x, 1), 0)
-0.5

```

> [@rpgowers](#):
>
> why does ForwardDiff work on the `sinc` function when it also contains a special point similar to this at x=0?

This has an explicit rule [in DiffRules.jl](https://github.com/JuliaDiff/DiffRules.jl/blob/9030629bbea6b25851789af5f236f35c9009b1f6/src/rules.jl#L60). Note that ForwardDiff reads these rules once while loading, so you can’t easily add rules for your functions this way. You can (as mentioned) add methods like `sinc(x::Dual)` :

```julia
julia> ForwardDiff.derivative(sinc, 0.0) # this calls cosc
-0.0

julia> @which sinc(ForwardDiff.Dual(0, 1))
sinc(d::Dual{T}) where T
     @ ForwardDiff ~/.julia/dev/ForwardDiff/src/dual.jl:238

julia> sinc1(x) = sin(x)/x;
julia> sinc2(x) = x==0 ? one(x) : sin(x)/x;
julia> sinc3(x) = abs2(x) > eps() ? sin(x)/x : 1 - x^2/6 + x^4/120;

julia> ForwardDiff.derivative(sinc1, 0.0)
NaN

julia> ForwardDiff.derivative(sinc2, 0.0) # was 0.0 on ForwardDiff v0.10
NaN

julia> ForwardDiff.derivative(sinc3, 0.0) # thanks to taylor series
0.0

julia> using ForwardDiff: Dual, value, partials

julia> sinc4(x::Dual{Z}) where Z = Dual{Z}(value(x), cosc(value(x)) * partials(x));

julia> ForwardDiff.derivative(sinc4, 0.0)
-0.0

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [January 30, 2023, 5:12pm UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780/4 "2023-01-30T17:12:15Z")

</div>

This is mostly unrelated to the ForwardDiff question, but for things like this `expm1` is very much your friend.

---

<div class="post-metadata">

**Author:** ![rpgowers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rpgowers/32/46229_2.png) [@rpgowers](https://discourse.julialang.org/u/rpgowers)\
**Post date:** [February 1, 2023, 9:49am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780/5 "2023-02-01T09:49:38Z")

</div>

This is interesting, thank you I have used a Taylor series approach to resolve this issue for now. Is there any particular advantage in defining `function(x::Dual{Z}) where ...` as opposed to/in addition to `function(x) = abs(x) > eps() ? ...`? Ultimately the definition of `cosc(x)` still requires a value to be defined at x=0, right?

---

<div class="post-metadata">

**Author:** ![rpgowers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rpgowers/32/46229_2.png) [@rpgowers](https://discourse.julialang.org/u/rpgowers)\
**Post date:** [February 1, 2023, 9:50am UTC](https://discourse.julialang.org/t/automatic-differentiation-of-function-with-special-points/93780/6 "2023-02-01T09:50:06Z")

</div>

Indeed it is, thanks for reminding me of this.