# Abs in JuMP - how (why?) does it work

**URL:** <https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [May 6, 2024, 2:29pm UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909 "2024-05-06T14:29:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![blob](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@blob](https://discourse.julialang.org/u/blob)\
**Post date:** [May 6, 2024, 2:29pm UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/1 "2024-05-06T14:29:30Z")

</div>

I cannot get my head around what JuMP/Ipopt is doing if I have an absolute value in the constraints. This runs and I cannot find out why:

```julia
using JuMP, Ipopt
tf=10

model = Model(Ipopt.Optimizer)

@variable(model, 0<=v) 
@variable(model, x[1:tf]) 
@objective(model, Max, v)
@constraint(model, tryErr[k=1:tf], abs(x[k]).>=v) ###This is equivalent to OR constraints: x[k]>=v OR x[k]<=-v, how does JuMP handle it?
@constraint(model, importantCstr[k=1:tf], abs.(10.0./k.*sin(x[k])).<=0.5)###This one is the same as two constraints: -0.5<=10.0./k.*sin(x[k]) AND 10.0./k.*sin(x[k])<=0.5, OK

optimize!(model)

```

I found [this topic](https://discourse.julialang.org/t/how-can-i-get-the-maximum-of-an-absolute-value-of-an-objective/101356/9) introducing binary variables - but if I have binary variables, then I cannot use Ipopt. So how is JuMP handling the `tryErr` constraint if the code runs?

The actual problem I am trying to solve can be interpreted as finding a maximum `v` that pushes all elements of `x` away from zero while keeping the `importantCstr` is still satisfied. The problem is nonlinear, so using MILP solvers is out of question.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 6, 2024, 6:51pm UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/2 "2024-05-06T18:51:27Z")

</div>

Ipopt is treating `abs` as a nonlinear function. It computes a derivative by looking at whether the child is positive or negative. The non-smoothness often causes convergence issues for Ipopt.

When you have `abs(x) <= 0.5`, this constraint is convex, and things should mostly work. Although for some problems Ipopt will not converge because it oscillates around `x = 0`.

When you have `abs(x) >= v`, this constraint is _non_-convex. If Ipopt converges, it may be to a local solution. If I had to guess, it will probably be related to the starting point, and shoot off to whichever “side” of `abs` is active first.

---

<div class="post-metadata">

**Author:** ![jbytecode](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbytecode/32/17719_2.png) [@jbytecode](https://discourse.julialang.org/u/jbytecode)\
**Post date:** [May 6, 2024, 7:20pm UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/3 "2024-05-06T19:20:09Z")

</div>

how about using smooth approximations of absolute value function? does it change something?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 6, 2024, 8:00pm UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/4 "2024-05-06T20:00:23Z")

</div>

That would fix the derivative/oscillating issue, not the non-convexity of `smooth_abs(x) >= v`.

---

<div class="post-metadata">

**Author:** ![jd-foster](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jd-foster/32/35824_2.png) [@jd-foster](https://discourse.julialang.org/u/jd-foster)\
**Post date:** [May 7, 2024, 4:13am UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/5 "2024-05-07T04:13:19Z")

</div>

The [“Huber loss”](https://en.wikipedia.org/wiki/Huber_loss?wprov=sfti1) is sometimes used along these lines.

---

<div class="post-metadata">

**Author:** ![blob](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@blob](https://discourse.julialang.org/u/blob)\
**Post date:** [May 7, 2024, 6:05am UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/6 "2024-05-07T06:05:44Z")

</div>

I am OK with local solutions (my original problem doesn’t have a unique solution for `x` anyway, it _may_ have a unique `v`). I was expecting the code to fail because I couldn’t figure out how the non-convex and non-smooth constraint could be reformulated into a smooth one (the non-smooth and convex is easier). I thought JuMP was doing something smart that I was missing 🙂 If ipopt simply treats `abs(x)` as a nonlinear function, the non-smoothness at zero seems to have little influence as I am pushing away from zero.

I figured out a reformulation like this:

```julia
@constraint(model, tryErrMul[k=1:tf], (x[k]-v)*(x[k]+v).>=0.0)

```

but I will read up on Huber loss, it looks interesting.

---

<div class="post-metadata">

**Author:** ![jbytecode](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbytecode/32/17719_2.png) [@jbytecode](https://discourse.julialang.org/u/jbytecode)\
**Post date:** [May 7, 2024, 6:22am UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/7 "2024-05-07T06:22:22Z")

</div>

I would be happy to see what happens when we use

\lambda(x) = \frac{xe^{kx} - xe^{-kx}}{e^{kx} + e^{-kx}}

instead of the `abs()` function. As @odow states, you will catch the oscillating issue. You also prevent staying near to zero by adding extra contraints also.

Edit: A good discussion on smoothing abs: [calculus - Approximate $|x|$ with a smooth function - Mathematics Stack Exchange](https://math.stackexchange.com/questions/728094/approximate-x-with-a-smooth-function)

Edit 2:

> However you lose the convexity.

---

<div class="post-metadata">

**Author:** ![blob](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@blob](https://discourse.julialang.org/u/blob)\
**Post date:** [May 8, 2024, 9:25am UTC](https://discourse.julialang.org/t/abs-in-jump-how-why-does-it-work/113909/8 "2024-05-08T09:25:54Z")

</div>

Just playing with the reformulation, here is the code:

```julia
using JuMP, Ipopt
tf=10
kVal=10

model = Model(Ipopt.Optimizer)
@variable(model, 0<=v) 
@variable(model, x[1:tf]) 
@objective(model, Max, v)
@expression(model, λ[k=1:tf], (x[k]*exp(x[k]*kVal)-x[k]*exp(-x[k]*kVal))./(exp(x[k]*kVal)+exp(-x[k]*kVal)))
@constraint(model, [k=1:tf], abs.(10.0./k.*sin(x[k])).<=0.5)
@constraint(model, tryErrNew[k=1:tf], λ[k].>=v) 
optimize!(model)

```

Unfortunately, this reformulation requires me to set the value of `kVal` that directly affects how well the reformulation approximates the absolute value. For my problem, it is important to make sure that for a given v if x^\* solves |x|-v=0, then \lambda(x^\*)-v=0 as well. And depending on the chosen `kVal`, the proposed reformulation \lambda(x) underestimates |x| around zero and the solutions to \lambda(x)-v=0 don’t overlap with the solutions to |x|-v=0. Because I don’t know the actual value of v, I cannot guarantee that the `kVal` will be large enough.

Besides, the proposed reformulation (or my implementation thereof) seems to mess something up numerically when I rewrite also the convex non-smooth constraint:

```julia
using JuMP, Ipopt
tf=10
kVal=10

model = Model(Ipopt.Optimizer)
@variable(model, 0<=v) 
@variable(model, x[1:tf]) 
@objective(model, Max, v)
@expression(model, λ[k=1:tf], (x[k]*exp(x[k]*kVal)-x[k]*exp(-x[k]*kVal))./(exp(x[k]*kVal)+exp(-x[k]*kVal)))
@constraint(model, [k=1:tf], 10.0./k.*sin(x[k]).<=0.5)
@constraint(model, [k=1:tf], 10.0./k.*sin(x[k]).>=-0.5)
@constraint(model, tryErrNew[k=1:tf], λ[k].>=v) 
optimize!(model)

```

gives  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/d/c/dcb142a62397ca526986a4dd42d273eaaebce605.png)  
whereas

```julia
using JuMP, Ipopt
tf=10
kVal=10

model = Model(Ipopt.Optimizer)
@variable(model, 0<=v) 
@variable(model, x[1:tf]) 
@objective(model, Max, v)
@constraint(model, tryErr[k=1:tf], abs(x[k]).>=v) 
@constraint(model, [k=1:tf], 10.0./k.*sin(x[k]).<=0.5)
@constraint(model, [k=1:tf], 10.0./k.*sin(x[k]).>=-0.5)
optimize!(model)

```

gives:  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/b/8b59af107d1d915a3353d1e2d13ff8e6447e2d75.png)