# How is the max operator dealt with in DualNumbers.jl?

**URL:** <https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743>\
**Category:** Numerics\
**Tags:** differentiation\
**Created:** [September 6, 2017, 12:11pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743 "2017-09-06T12:11:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vgdev](https://avatars.discourse-cdn.com/v4/letter/v/47e85d/32.png) [@vgdev](https://discourse.julialang.org/u/vgdev)\
**Post date:** [September 6, 2017, 12:11pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743/1 "2017-09-06T12:11:21Z")

</div>

I am minimizing an objective function using a local optimization algorithm and I am supplying it with the gradient generated by ForwardDiff. However, my objective function contains functionals including the max(x,0) operator, which is non-differentiable at x =0. Experiments shows that the forwarddiff gradient and the finite difference gradient agrees, and I am able to find the minimum. However, how is the max operator dealt with in `DualNumbers.jl`? I did not find anything about it in the [source](https://github.com/JuliaDiff/DualNumbers.jl/blob/master/src/dual.jl). Is it smoothed somehow?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 6, 2017, 1:24pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743/2 "2017-09-06T13:24:19Z")

</div>

> [@vgdev](#):
>
> However, my objective function contains functionals including the max(x,0) operator

You should really re-write your optimization problem to be differentiable, since otherwise your optimizer might get stuck (or at least have very slow convergence). In problems involving `max` or `min`, you can generally do this by a standard trick involving adding dummy variables and constraints.

See, for example: [NLopt Introduction - NLopt Documentation](http://nlopt.readthedocs.io/en/latest/NLopt_Introduction/#equivalent-formulations-of-optimization-problems)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 6, 2017, 1:28pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743/3 "2017-09-06T13:28:26Z")

</div>

> [@vgdev](#):
>
> However, how is the max operator dealt with in DualNumbers.jl?

I think it just computes a one-sided derivative. i.e. it ignores the discontinuity (just picking the derivative from one side or the other of the kink in the `max` function). See [max when values agree? · Issue #53 · JuliaDiff/DualNumbers.jl · GitHub](https://github.com/JuliaDiff/DualNumbers.jl/issues/53)

---

<div class="post-metadata">

**Author:** ![vgdev](https://avatars.discourse-cdn.com/v4/letter/v/47e85d/32.png) [@vgdev](https://discourse.julialang.org/u/vgdev)\
**Post date:** [September 6, 2017, 1:36pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743/4 "2017-09-06T13:36:06Z")

</div>

So if I actually never evaluate max(x,0) at x =0, i should be okay?

---

<div class="post-metadata">

**Author:** ![antoine-levitt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/antoine-levitt/32/4008_2.png) [@antoine-levitt](https://discourse.julialang.org/u/antoine-levitt)\
**Post date:** [September 6, 2017, 1:40pm UTC](https://discourse.julialang.org/t/how-is-the-max-operator-dealt-with-in-dualnumbers-jl/5743/5 "2017-09-06T13:40:34Z")

</div>

If your optimizer always stays in the region where x is always positive (or negative), you should be fine. If it jumps around then you might be in trouble (in particular, quasi-Newton algorithms should get very confused)