# Autodiff removing singularity

**URL:** <https://discourse.julialang.org/t/autodiff-removing-singularity/109498>\
**Category:** Data\
**Tags:** forwarddiff, autodiff\
**Created:** [January 31, 2024, 7:06am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498 "2024-01-31T07:06:26Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [January 31, 2024, 7:06am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/1 "2024-01-31T07:06:26Z")

</div>

I have a function whose derivative isn’t defined at the origin, but is known ahead of time when approaching from x\>0, which is the limit I want. How can I add a special case to the ForwardDiff dual such that it uses this value at the origin but uses the default process elsewhere?

I am sure this has been asked before, but I m must be using the wrong key words

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [January 31, 2024, 8:19am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/2 "2024-01-31T08:19:24Z")

</div>

Something like this for the absolute value? Inspired by the [ForwardDiff docs](https://juliadiff.org/ForwardDiff.jl/stable/dev/how_it_works/)

```julia
julia> using ForwardDiff: Dual, derivative, value, partials

julia> myabs(x::Real) = abs(x)
myabs (generic function with 1 method)

julia> function myabs(d::Dual{T}) where {T}
           x, Δx = value(d), partials(d)
           slope = ifelse(x <= zero(x), -one(x), one(x)) # note the <=
           return Dual{T}(x, slope * Δx) 
       end
myabs (generic function with 2 methods)

julia> derivative(abs, 0)
1

julia> derivative(myabs, 0)
-1

```

Note that in general, automatic differentiation applies to programs, not functions, and so the behavior at nondifferentiable points is implementation-dependent.  
See this article for details:

> [_A mathematical model for automatic differentiation in machine learning_](https://arxiv.org/abs/2006.02080), Bolte & Pauwels (2020)

---

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [January 31, 2024, 9:33am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/3 "2024-01-31T09:33:08Z")

</div>

Thanks! Your code defines the derivative everywhere, but I only want to over-write the normal behavior at the origin. I guess I could try…

```julia
_myfun(x) = abs(x) # real function is more complicated
myfun(x) = _myfun(x)
myfun(d::Dual{T}) where {T} = ifelse(value(d) == zero(value(d)), Dual{T}(value(d), 2pi * partials(d)), _myfun(d))

```

If I don’t wrap the function, I don’t see what I could call in the “else” part, but I guess this works…

---

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [January 31, 2024, 9:56am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/4 "2024-01-31T09:56:39Z")

</div>

Oh no, another wrinkle! My real function has other arguments. When I call `derivative`, I make an anonymous function. How am I supposed to handle that?

```julia
myfun(x,y) = abs(x-y) # but harder
derivative(x->myfun(x,y),y) # want to fix behavior at x==y

```

I guess, I’ll need to work backwards through the function chain until I get to the _actual_ singularity…

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [January 31, 2024, 10:15am UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/5 "2024-01-31T10:15:00Z")

</div>

Guess you have to overload `myfun` for both inputs being `Dual`

---

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [January 31, 2024, 2:46pm UTC](https://discourse.julialang.org/t/autodiff-removing-singularity/109498/6 "2024-01-31T14:46:31Z")

</div>

Just for future reference. My real function had a vector input `x` and additional arguments such as a reference location `y` and direction `n`. The basic solution method is the same, but I had to handle the partials of `d` in a way general enough to handle both `gradient(x->ϕ(x,y,n),x)` and `derivative(t->ϕ(x+t*n,y,n),0)`-type calls.

```julia
_ϕ(x,y,n) = # iterative vector function with scalar output
ϕ(x,y,n) = _ϕ(x,y,n)
function ϕ(d::Vector{<:Dual{Tag}},y,n) where Tag
    x,Δx = value.(d),stack(partials.(d))
    ifelse(x == y, Dual{Tag}(_ϕ(x,y,n),2π*Δx*n...), _ϕ(d,y,n))
end

```