# JuMP error for nonlinear optimization

**URL:** <https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983>\
**Category:** Optimization (Mathematical)\
**Tags:** jump, nonlinear\
**Created:** [November 5, 2021, 12:32am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983 "2021-11-05T00:32:16Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 12:32am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/1 "2021-11-05T00:32:16Z")

</div>

I’m trying to set up a nonlinear optimization problem and I’m getting an error I don’t understand…

```julia
import JuMP as J
import Ipopt

function smoothabs(x) # Smooth approximation to abs(x)
    k = 1000
    f = 2/k
    f*log(1+exp(k*x)) - x - f*log(2)
end
smoothmin2(x,y) = (x + y - smoothabs(x-y))/2 # Smooth approximation to min(x,y)

# nc and nb are integers, both about 200 to 300
# A is a Float64 matrix of ones and zeros of dimension [nc, nb]
# abw is a Float64 approximately equal to 2000.0

model = J.Model(Ipopt.Optimizer)
J.register(model, :smoothmin, 2, smoothmin2, autodiff=true)
J.@variable(model, 0 <= cbw[i=1:nc]) 
J.@NLconstraint(model, con1, sum(c for c in cbw) ≤ abw)
J.@NLobjective(model, Max, smoothmin(sum(A[1,n] * cbw[n] for n in 1:nc), dmndvec[1]))
J.@NLobjective(model, Max, sum(s for s in (smoothmin2(sum(A[m,n]*cbw[n] for n in 1:nc), dmndvec[m]) for m in 1:nb)))
@time J.optimize!(model)

```

The problem is nonlinear because I am using the function `smoothmin2` to (approximately) choose the smaller of two arguments, while remaining differentiable so I can still use `JuMP`.

This produces the following error message:

```julia
exp is not defined for type GenericAffExpr. Are you trying to build a nonlinear problem? Make sure you use @NLconstraint/@NLobjective.

```

My constraints and objective are all specified using `@NL...`, so why am I getting this error message?

Thanks in advance for any help.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 5, 2021, 12:51am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/2 "2021-11-05T00:51:34Z")

</div>

Your second objective calls `smoothmin2`, not the registered function `smoothmin`.

Your expressions are all algebraic, so you should try writing it out without using user-defined functions if possible. If the model contains user-defined functions, we disable second-derivative information, so. the performance is worse.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 1:31am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/3 "2021-11-05T01:31:36Z")

</div>

> [@odow](#):
>
> tains user-defined functions, we disable sec

Thanks, I guess I don’t understand the `register` function. I thought that if I register the symbol `:smoothmin` to refer to the Julia function `smoothmin2` then I could use `smoothmin` within the `NLobjective` call to refer to the `smoothmin2` function. Is that incorrect?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 5, 2021, 1:50am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/4 "2021-11-05T01:50:32Z")

</div>

> Is that incorrect?

Yes, but you have

```Julia
or s in (smoothmin2(sum(A[m,n]*

```

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 2:11am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/5 "2021-11-05T02:11:16Z")

</div>

Ah, thanks! Guess I’ve been staring at it too long. Now, after correcting (and removing one of the constraints):

```julia
model = J.Model(Ipopt.Optimizer)
J.register(model, :smoothmin, 2, smoothmin2, autodiff=true)
J.@variable(model, 0 <= cbw[i=1:nc]) 
J.@NLobjective(model, Max, sum(s for s in (smoothmin(sum(A[m,n]*cbw[n] for n in 1:nc), dmndvec[m]) for m in 1:nb)))
@time J.optimize!(model)

```

I get this error:

```julia
UndefVarError: smoothmin not defined

```

I tried restarting Julia to no avail. I then tried using the symbol `:smoothmin2` to represent the function `smoothmin2`:

```julia
model = J.Model(Ipopt.Optimizer)
J.register(model, :smoothmin2, 2, smoothmin2, autodiff=true)
J.@variable(model, 0 <= cbw[i=1:nc]) 
J.@NLobjective(model, Max, sum(s for s in (smoothmin2(sum(A[m,n]*cbw[n] for n in 1:nc), dmndvec[m]) for m in 1:nb)))
@time J.optimize!(model)

```

and I get the error:

```julia
exp is not defined for type GenericAffExpr. Are you trying to build a nonlinear problem? Make sure you use @NLconstraint/@NLobjective.

```

Any idea what I’m doing wrong?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 5, 2021, 2:58am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/6 "2021-11-05T02:58:17Z")

</div>

> [@PeterSimon](#):
>
> `sum(s for s in (smoothmin(sum(A[m,n]*cbw[n] for n in 1:nc), dmndvec[m]) for m in 1:nb)))`

I think you mean:

```julia
sum(smoothmin(sum(A[m,n]*cbw[n] for n in 1:nc), dmndvec[m]) for m in 1:nb))

```

The `sum(s for s in X)` causes us to try and evaluate `S` as a constant, not part of the decision variables. This _doesn’t_ go through the function registration process.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 3:13am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/7 "2021-11-05T03:13:43Z")

</div>

Thanks! That allowed the optimizer to begin. After reinserting the necessary constraint Ipopt terminates with this error:

```julia
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit https://github.com/coin-or/Ipopt
******************************************************************************

This is Ipopt version 3.13.4, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...: 0
Number of nonzeros in inequality constraint Jacobian.: 354
Number of nonzeros in Lagrangian Hessian.............: 0

Total number of variables............................: 354
                     variables with only lower bounds: 354
                variables with lower and upper bounds: 0
                     variables with only upper bounds: 0
Total number of equality constraints.................: 0
Total number of inequality constraints...............: 1
        inequality constraints with only lower bounds: 0
   inequality constraints with lower and upper bounds: 0
        inequality constraints with only upper bounds: 1

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
   0 1.6614353e+01 0.00e+00 1.74e+01 0.0 0.00e+00 - 0.00e+00 0.00e+00 0
Warning: Cutting back alpha due to evaluation error
   1 6.2761881e+03 0.00e+00 1.62e+01 0.9 2.68e+03 - 6.94e-02 4.98e-01f 2
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   2 7.9405496e+03 0.00e+00 1.70e+01 0.6 1.38e+03 - 4.48e-01 1.22e-01f 4
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   3 8.7542582e+03 0.00e+00 1.68e+01 -5.2 1.20e+03 - 2.48e-01 6.16e-02f 5
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   4 8.8470353e+03 0.00e+00 1.73e+01 0.0 1.28e+03 - 8.83e-01 6.78e-03f 8
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   5 8.8912916e+03 0.00e+00 1.71e+01 0.3 1.31e+03 - 6.77e-01 3.28e-03f 9
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   6 8.9322177e+03 0.00e+00 1.73e+01 0.2 1.38e+03 - 8.40e-01 3.11e-03f 9
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
   7 8.9699505e+03 0.00e+00 1.71e+01 0.3 1.19e+03 - 3.86e-01 3.57e-03f 9
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error
Warning: Cutting back alpha due to evaluation error

Number of Iterations....: 7

Number of objective function evaluations = 58
Number of objective gradient evaluations = 9
Number of equality constraint evaluations = 0
Number of inequality constraint evaluations = 58
Number of equality constraint Jacobian evaluations = 0
Number of inequality constraint Jacobian evaluations = 9
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 1.386
Total CPU secs in NLP function evaluations = 0.190

EXIT: Invalid number in NLP function or derivative detected.
 14.603243 seconds (74.75 M allocations: 4.373 GiB, 5.11% gc time, 1.15% compilation time)

```

Is this error related to the fact that second derivatives are not available? I would like to follow your suggestion and eliminate the registered function, but I don’t see how to do that. I apologize for leaning on your help so heavily.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 5, 2021, 4:19am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/8 "2021-11-05T04:19:14Z")

</div>

```julia
using JuMP
import Ipopt
nc, nb = 200, 300
A = rand(0:1, nb, nc)
abw = 2.0
dmndvec = rand(nb)
model = Model(Ipopt.Optimizer)
@variable(model, cbw[1:nc] >= 0)  
@constraint(model, sum(cbw) <= 2.0)
@expression(model, Ax[m=1:nb], sum(A[m, n] * cbw[n] for n in 1:nc))
function smooth_min(model, x, y)
    k = 1_000
    f = 2 / k
    return @NLexpression(
        model, 
        (x + y - f * log(1 + exp(k * (x - y))) - (x - y) - f * log(2)) / 2,
    )
end
nl_expr = smooth_min.(model, Ax, dmndvec)
@NLobjective(model, Max, sum(nl_expr[m] for m in 1:nb))
optimize!(model)

```

but…

`exp` is not defined in `Float64` arithmetic for values \>709.

```nohighlight
julia> exp(709.0)
8.218407461554972e307

julia> exp(710.0)
Inf

```

Your model has `1000 * (x - y)`, so this approximation is only valid for `x` close to `y`. If `x > y + 1`, then Ipopt will evaluate an `Inf`.

I suggest you reconsider your modeling approximation.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 11:54am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/9 "2021-11-05T11:54:32Z")

</div>

Wow! I had no idea that one could return a `@NLexpression` from a function, nor that one could generate a vector of same in such a fashion. It’s beautifully concise. I will experiment with this and work on a better smooth approximation to the `min` function, then report back before marking your answer as the solution. Again, thanks for the detailed response.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [November 5, 2021, 12:50pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/10 "2021-11-05T12:50:04Z")

</div>

the solution here is to use [LogExpFunctions · LogExpFunctions.jl](https://juliastats.org/LogExpFunctions.jl/stable/#LogExpFunctions.log1pexp) it computes `log(1+exp(x))` without overflow.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 1:04pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/11 "2021-11-05T13:04:08Z")

</div>

Thanks. This approximation also seems to work for my application:

```julia
smooth_min(model, x, y) = @NLexpression(model, (x + y - sqrt((x-y)^2 + 1e-8))/2)

```

---

<div class="post-metadata">

**Author:** ![cvanaret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cvanaret/32/11594_2.png) [@cvanaret](https://discourse.julialang.org/u/cvanaret)\
**Post date:** [November 5, 2021, 4:58pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/12 "2021-11-05T16:58:57Z")

</div>

Note (in case you need it): you can reformulate your problem into a smooth problem (without max or abs) as follows:  
if the objective function contains |x|, replace it with p+n, where p and n are additional variables, and add the following constraints:  
x = p-n  
p, n \ge 0  
Ta-da! You now have a smooth problem.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 10:13pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/13 "2021-11-05T22:13:13Z")

</div>

Great trick! I’m sure there must be lots of other tricks for formulating optimizations. Are they listed somewhere?

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 10:24pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/14 "2021-11-05T22:24:41Z")

</div>

As a final followup, the code is now working well, thanks to your extensive help. ~~For the record, I did have to change `@expression(model, Ax[m=1:nb], sum(A[m, n] * cbw[n] for n in 1:nc))`  
to `@NLexpression(model, Ax[m=1:nb], sum(A[m, n] * cbw[n] for n in 1:nc))`  
to get JuMP to accept it, I believe because it is used in the call to `smooth_min`.~~

Edit: As pointed out in a later post in this thread, recent versions of JuMP do not require this.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 5, 2021, 11:22pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/15 "2021-11-05T23:22:51Z")

</div>

> I’m sure there must be lots of other tricks for formulating optimizations. Are they listed somewhere?

The Mosek Modeling Cookbook has a bunch: [MOSEK Modeling Cookbook — MOSEK Modeling Cookbook 3.3.0](https://docs.mosek.com/modeling-cookbook/index.html). But optimization is more of an art than a science.

> For the record, I did have to change

Update your version of JuMP. This should be allowed in any version of JuMP 0.21.8 or later.

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 5, 2021, 11:28pm UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/16 "2021-11-05T23:28:12Z")

</div>

Thanks for lead on the tips, I’m going to enjoy reading them!

`Pkg.status()` says my version of JuMP is 0.21.10.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 6, 2021, 1:17am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/17 "2021-11-06T01:17:59Z")

</div>

Do you have a reproducible example? Using `@expression` works fine for me.

```nohighlight
julia> using JuMP

julia> import Ipopt

julia> nc, nb = 200, 300;

julia> A = rand(0:1, nb, nc);

julia> abw = 2.0;

julia> dmndvec = rand(nb);

julia> model = Model(Ipopt.Optimizer);

julia> @variable(model, cbw[1:nc] >= 0);

julia> @constraint(model, sum(cbw) <= 2.0);

julia> @expression(model, Ax[m=1:nb], sum(A[m, n] * cbw[n] for n in 1:nc));

julia> smooth_min(model, x, y) = @NLexpression(model, (x + y - sqrt((x-y)^2 + 1e-8))/2);

julia> nl_expr = smooth_min.(model, Ax, dmndvec);

julia> @NLobjective(model, Max, sum(nl_expr[m] for m in 1:nb));

julia> optimize!(model)
This is Ipopt version 3.13.4, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...: 0
Number of nonzeros in inequality constraint Jacobian.: 200
Number of nonzeros in Lagrangian Hessian.............: 20100

Total number of variables............................: 200
                     variables with only lower bounds: 200
                variables with lower and upper bounds: 0
                     variables with only upper bounds: 0
Total number of equality constraints.................: 0
Total number of inequality constraints...............: 1
        inequality constraints with only lower bounds: 0
   inequality constraints with lower and upper bounds: 0
        inequality constraints with only upper bounds: 1

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
   0 1.4728209e+02 0.00e+00 2.81e+00 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
[...]
  66 1.4791869e+02 0.00e+00 3.06e-09 -8.6 9.28e-04 - 1.00e+00 1.00e+00h 1

Number of Iterations....: 66

                                   (scaled) (unscaled)
Objective...............: -8.6502173739675257e+01 1.4791868520649248e+02
Dual infeasibility......: 3.0603609204815344e-09 5.2332160458452213e-09
Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00
Complementarity.........: 2.5059036087747929e-09 -4.2850942472230598e-09
Overall NLP error.......: 3.0603609204815344e-09 5.2332160458452213e-09

Number of objective function evaluations = 191
Number of objective gradient evaluations = 67
Number of equality constraint evaluations = 0
Number of inequality constraint evaluations = 191
Number of equality constraint Jacobian evaluations = 0
Number of inequality constraint Jacobian evaluations = 1
Number of Lagrangian Hessian evaluations = 66
Total CPU secs in IPOPT (w/o function evaluations) = 0.479
Total CPU secs in NLP function evaluations = 9.020

EXIT: Optimal Solution Found.

(ipopt) pkg> st
      Status `/private/tmp/ipopt/Project.toml`
  [b6b21f68] Ipopt v0.7.0
  [4076af6c] JuMP v0.21.10

```

---

<div class="post-metadata">

**Author:** ![PeterSimon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petersimon/32/25193_2.png) [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)\
**Post date:** [November 6, 2021, 2:21am UTC](https://discourse.julialang.org/t/jump-error-for-nonlinear-optimization/70983/18 "2021-11-06T02:21:25Z")

</div>

It works for me now as well. I think that while I was experimenting with installing/uninstalling different solvers, JuMP must have been updated and I didn’t notice it. I’ve edited my previous post to show that the change to `@NLexpression` is not needed.