# Nonlinear optimization: Registering a function with parameters

**URL:** <https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [June 16, 2023, 4:46pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457 "2023-06-16T16:46:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 16, 2023, 4:46pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/1 "2023-06-16T16:46:42Z")

</div>

I followed [the link](https://jump.dev/JuMP.jl/stable/manual/nlp/#Register-a-function) that teaches how to register a function in a NL model.

As far as I can see, the registered functions’ inputs are optimization variables. I would like to also allow inputting a separate object, whose values would define what the function does. Is there a way to not define the functions separately, but rather register one function that takes the so-called ‘context’ parameters?

For example, I would like to define a function `evaluate_f!(f, x)` where `x` is the optimization variable (which is fine by JuMP), and `f` is an object that I defined. For example, if f is of type `Quadratic`, then it has attributes of its data `Q, q` that define x^\top Q x + q^\top x. So the function that I register will look like:

```julia
function evaluate_f!(f::Quadratic, x)
    n = length(f.q)
    0.5*sum(f.Q[i, j] * x[i] * x[j] for i in 1:n, j in 1:n)+sum(f.q[i] * x[i] for i in 1:n)
end

```

for which I am using the scalar `i,j` approach rather than vector notation for JuMP’s nonlinear solver interactions. Similarly, there will be different versions for, _e.g._, `f::Linear`.

The reason why I want to do this is that I would like to optimize functions that look like \log(\exp(f(x)) + \exp(g(x))) where instead of defining f, g, ... separately I can automatically keep all my functions in a list `fn_collection` that keeps all my many functions and

```cpp
@NLexpression(model, exp_to_minimize, log(sum(exp(evaluate_f!(fn_collection[k], x)) for k in 1:length(fn_collection))))

```

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [June 17, 2023, 1:37pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/2 "2023-06-17T13:37:20Z")

</div>

Perhaps I am missing the issue, but doesn’t

```julia
for fn in fn_collection
  @NLexpression(model, exp_to_minimize, x -> log(sum(exp(evaluate_f!(fn, x)))))
end

```

produce the desired result?

I am assuming you want to generate one expression at a time; not a vector of expressions.

Side note: there are efficient / accurate ways of computing `log(sum(exp()))` in some packages (which I cannot remember of the top of my head).

BTW: why `evaluate_f!`, suggesting a mutating function?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 19, 2023, 12:14am UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/3 "2023-06-19T00:14:04Z")

</div>

The short answer is that you cannot register a function which accepts something other than scalar decision variables. (This is something else that will be fixed in the rewrite of the JuMP nonlinear interface.)

---

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 19, 2023, 11:27am UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/4 "2023-06-19T11:27:33Z")

</div>

Thank you for your kind reply, Oscar! Do you think I can still hard-code this (something like @hendri54 kindly suggested, which I will reply to soon, but ends up not working as expected).

**Update:** I have done the following slightly brute-forced method:

```julia
for fn_nr in 1:length(f_collection)
    # println(typeof(fn))
    try 
        register(model, Symbol(:expr, fn_nr), 1, x -> scalar_expr!(f_collection[fn_nr], x); autodiff = true)
    catch e
        if occursin("already", e.msg)
            println("Already registered, skipping.")
        else
            throw(e)
        end
    end
end

```

Although this let me registers more than one function, I still get `Unexpected array` errors suggesting `x` also cannot be given as a vector.

**Update 2:** Nope, I tried everything. Scalar definition of `x...` as suggested on JuMP page and `x -> scalar_expr!(f_collection[fn_nr], x)` kind of definition contradicts. I will update if I solve this issue.

---

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 19, 2023, 4:11pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/5 "2023-06-19T16:11:59Z")

</div>

Thank you very much for your kind reply!

The reason I have `!` is just to remember that I have written the function myself, it is just my own convention, and it indeed does contradict the common understanding of mutating function.

Except for that, your suggestion makes a lot of sense to me. But my functions appear _inside_ logsumexp, hence I need to access the `evaluate_f!` functions within a single logsumexp function. I don’t know how to circumvent this problem.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 19, 2023, 7:57pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/6 "2023-06-19T19:57:56Z")

</div>

Untested, because I don’t have a reproducible example, but this should point you in the right direction:

```julia
evaluate_f!(f::Linear, x) = f.q' * x
evaluate_f!(f::Quadratic, x) = 0.5 * x' * f.Q * x + f.q' * x
model = Model()
@variable(model, x[1:4])
f_x = Any[]
for (k, f_k) in enumerate(fn_collection)
    f = (x...) -> evaluate_f!(f_k, collect(x))
    register(model, Symbol("f_$k"), length(x), f; autodiff = true)
    expr = Expr(:call, Symbol("f_$k"), x...)
    push!(f_x, add_nonlinear_expression(model, expr))
end
@NLexpression(model, exp_to_minimize, log(sum(exp(f_x_k) for f_x_k in f_x)))

```

See [Nonlinear Modeling · JuMP](https://jump.dev/JuMP.jl/stable/manual/nlp/#More-complicated-examples)

---

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 19, 2023, 11:04pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/7 "2023-06-19T23:04:09Z")

</div>

It indeed works! Thank you for this amazing support. I would have never thought about defining the mapping from `x... ->` as well as apply the `Expr` trick pre-`@NLexpression`. This is excellent!

This also allows me to optimize the `max` of these functions (instead of logsumexp) as — `@NLexpression(model, exp_to_minimize, max(f_x...))`

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 19, 2023, 11:11pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/8 "2023-06-19T23:11:48Z")

</div>

Yeah it’s a shame that the syntax is very convoluted and not at all obvious. JuMP can do a lot of things, but they can be hard to explain.

I’m a broken record at this point, but the nonlinear rewrite will fix all of this, and your original code will “just work.”

---

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 19, 2023, 11:22pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/9 "2023-06-19T23:22:20Z")

</div>

Haha, thank you :)! BTW, what I said “it indeed works” was for Ipopt. When I try BARON, I get:

`UnrecognizedExpressionException: unrecognized function call expression: f_1(x[1], x[2], x[3]) Stacktrace: [1] to_str(c::Expr) @ BARON [~/.julia/packages/BARON/xgWzt/src/util.jl:156](https://file+.vscode-resource.vscode-cdn.net/Users/arasselvi/Workspace/SumOfConvexMax/julia/quadraticsimplex/~/.julia/packages/BARON/xgWzt/src/util.jl:156) [2] (::BARON.var"#8#10")(d::Expr) @ BARON [./none:0](https://file+.vscode-resource.vscode-cdn.net/Users/arasselvi/Workspace/SumOfConvexMax/julia/quadraticsimplex/none:0)`

is this expected?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 20, 2023, 12:05am UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/10 "2023-06-20T00:05:04Z")

</div>

BARON doesn’t support user-defined functions. It writes the problem to a file, so it can’t write out arbitrary Julia code.

---

<div class="post-metadata">

**Author:** ![selvi-aras](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/selvi-aras/32/50637_2.png) [@selvi-aras](https://discourse.julialang.org/u/selvi-aras)\
**Post date:** [June 20, 2023, 12:08am UTC](https://discourse.julialang.org/t/nonlinear-optimization-registering-a-function-with-parameters/100457/11 "2023-06-20T00:08:52Z")

</div>

Thank you, great to know!