# Unable to use lagrange to solve 2 varable functions in modeling toolkit

**URL:** <https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043>\
**Category:** Optimization (Mathematical)\
**Created:** [November 25, 2021, 5:47am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043 "2021-11-25T05:47:13Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 5:47am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/1 "2021-11-25T05:47:13Z")

</div>

```julia
	@parameters x_1 x_2
	@variables u 

   eqs = [0 ~ u - (x_1+2)*(x_2+1), 130 ~ 4*x_1+6*x_2]

@named sys = NonlinearSystem(eqs, [x_1, x_2],[])

prob = NonlinearProblem(structural_simplify(sys),[x_1=>1.0,x_2=>1.0,u=>max])

```

generates an error, saying that there is no equation for x\_1

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 25, 2021, 8:31am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/2 "2021-11-25T08:31:16Z")

</div>

I am not yet perfectly familiar with ModelingToolkit, but just looking at your code, it seems that you are mixing the roles of a _parameter_ and a _variable_. You declare `x_1` and `x_2` as parameters but then you treat them as variables (as if you wanted to solve the problem with respect to them). Isn’t this the cause of the problem? Have a look at [Modeling Nonlinear Systems · ModelingToolkit.jl](https://mtk.sciml.ai/dev/tutorials/nonlinear/), they enter variables and parameters separately in NonlinearSystem() and NonlinearProblem().

---

<div class="post-metadata">

**Author:** ![tred](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@tred](https://discourse.julialang.org/u/tred)\
**Post date:** [November 25, 2021, 10:09am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/3 "2021-11-25T10:09:16Z")

</div>

Here is a working example:

```julia
using ModelingToolkit, NonlinearSolve

@variables x_1 x_2
@parameters u = 1.

eqs = [0 ~ u - (x_1+2)*(x_2+1), 130 ~ 4*x_1+6*x_2]

@named sys = NonlinearSystem(
    eqs,
    [x_1, x_2],
    [u],
    defaults = [
        x_1 => 1,
        x_2 => (130 - 4*x_1)/6
        ]
)

prob = NonlinearProblem(structural_simplify(sys), [])
solve(prob, NewtonRaphson())

```

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 4:59pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/4 "2021-11-25T16:59:48Z")

</div>

Thanks, I was confused on variables snd parameters, and thus cleared it up. Only thing is that I don’t know u , but I want yo maximize u.

---

<div class="post-metadata">

**Author:** ![tred](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@tred](https://discourse.julialang.org/u/tred)\
**Post date:** [November 25, 2021, 5:16pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/5 "2021-11-25T17:16:36Z")

</div>

You might want to check [GalacticOptim](https://galacticoptim.sciml.ai/stable/) and [OptimizationProblem in ModelingToolkit](https://mtk.sciml.ai/dev/tutorials/optimization/).

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 25, 2021, 5:26pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/6 "2021-11-25T17:26:08Z")

</div>

I’ll reiterate my suggestion to use JuMP instead:

> [@Advice on using ModelingToolkit to solve Lagrange Problems](https://discourse.julialang.org/t/advice-on-using-modelingtoolkit-to-solve-lagrange-problems/71473/25):
>
> julia\> using JuMP, Ipopt julia\> function solve() model = Model(Ipopt.Optimizer) @variable(model, x[1:2] \>= 0.001, start = 1) @NLobjective(model, Max, 16 \* log(x[1]) + 9 \* log(x[2])) @NLconstraint(model, x[1]^2 / 100 + x[2]^2 / 36 == 100) optimize!(model) @assert termination\_status(model) == LOCALLY\_SOLVED return value.(x) end solve (generic function with 1 method) julia\> solve() This is Ipopt version 3.13.4, ru…

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 5:26pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/7 "2021-11-25T17:26:19Z")

</div>

do i replace prob with:

```julia
prob = OptimizationProblem(structural_simplify(sys),[x_1=>1.0,x_2=>1.0,u=>1.0])

```

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 5:27pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/8 "2021-11-25T17:27:53Z")

</div>

I want to learn both, so that I can compare them. I’ve done this in JuMP and that is defiantly a valid option.

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 25, 2021, 5:40pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/9 "2021-11-25T17:40:45Z")

</div>

I am still not sure I understand what you are after. You write

> [@brett\_knoss](#):
>
> Only thing is that I don’t know u , but I want yo maximize u.

But then if `x_1` and `x_2` are known/given/fixed, and you impose the constraint `0 ~ u - (x_1+2)*(x_2+1)`, and there is not freedom left for you to do any optimization whatsoever. Your `u` is just given by the equation. Nothing to maximize here.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 5:42pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/10 "2021-11-25T17:42:27Z")

</div>

u(x\_1,x\_2)=(x\_1+2)\*(x\_2+1) u is meant to be a function of x\_1 and x\_2

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 25, 2021, 6:32pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/11 "2021-11-25T18:32:44Z")

</div>

I see. Only now I get it. To summarize, the full description of the problem is therefore this:

`maximize (x_1+2)*(x_2+1)`

`subject to 4*x_1+6*x_2 = 130 `

Concerning your attempted solving of this problem using `ModelingToolkit`, it appears to me that `NonlinearSystem()` is just an equation solver (it cannot do maximization) and the `OptimizationSystem()` cannot accept constraints. This is at least what I get from the docs, I hope I have not overlooked anything.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 25, 2021, 6:46pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/12 "2021-11-25T18:46:11Z")

</div>

It sounds like I’m better off using JuMP then.

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 26, 2021, 12:03am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/13 "2021-11-26T00:03:34Z")

</div>

By the way, if the particular structure of the problem is typical of your needs, this is a QP problem (a quadratic cost function and linear equality constraints). As such it can be solved without a general optimization package just by using a solver for linear equations (look the _KKT matrix_ up if you are not familiar with it, for example [here](https://web.stanford.edu/class/ee364a/lectures/equality.pdf)).

Here comes a code

```julia
# Parameterization of the quadratic cost function 1/2 x'Qx + r'x + c
Q = [0.0 1.0; 1.0 0.0]
r = [1.0, 2.0]
c = 2.0

# Parameterization of the linear constraint Ax = b
A = [4.0 6.0]
b = 130

# KKT linear system
K = [hcat(Q,A'); hcat(A, 0)]
d = vcat(-r,b)
y = K\d

```

The maximum is at

```julia
julia> xopt = y[1:2]
2-element Vector{Float64}:
 16.0
 11.0

```

That this is truly a maximum can be verified by checking negativity of the projected Hessian, which in this case can be computed using

```julia
julia> Z = nullspace(A);

julia> Z'*Q*Z
1×1 Matrix{Float64}:
 -0.923076923076923

```

We are lucky here because the Hessian of the uncostrained cost function is actually indefinite (the matrix `Q` has one positive and one negative eigval) and the unconstrained problem is unbounded. But for the contstrained problem we have a (unique and bounded) maximum.  
Finally, some plots

```julia
# Plotting the cost function, the constraint and the optimal solution
using Plots

f(x₁,x₂) = (x₁+2)*(x₂+1)

x₁ = range(-100.0,200.0,length=500)
x₂ = range(-100.0,120.0,length=500)

plot(x₁,x₂,f,st=:surface,xlabel="x₁",ylabel="x₂",zlabel="f(x₁,x₂)")

```

![fcn](https://global.discourse-cdn.com/julialang/original/3X/6/4/64252ad8eaf82719ca1c726a885d6dd9d7726110.png)

```julia
contour(x₁,x₂,f,xlabel="x₁",ylabel="x₂",levels=50)
x₂sol(x₁) = (130.0-4*x₁)/6
plot!(x₁,x₂sol,label="Admissible solutions")
plot!([y[1]],[y[2]],markershape=:circle,label="Optimal solution")

```

![contour](https://global.discourse-cdn.com/julialang/original/3X/7/f/7f12cf2798e8a7e93accef9acf53a15cdd698e9a.png)

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 26, 2021, 12:21am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/14 "2021-11-26T00:21:30Z")

</div>

What package are you using?

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 26, 2021, 12:29am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/15 "2021-11-26T00:29:12Z")

</div>

Package for what? I used nothing but the backslash for solving the linear equations (well, the code should have started with `using LinearAlgebra` because I also computed `nullspace`).

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 26, 2021, 3:52am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/17 "2021-11-26T03:52:29Z")

</div>

This makes sense, I’m not sure I know enough about linear algebra. I’m going through an Econ text now, and can find stuff about matrices in an earlier chapter. Are the matrices used Hessian matrices?

I get that Q is the null matirx, and A is a matrix of prices, but what do r and c mean?

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 26, 2021, 8:19am UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/18 "2021-11-26T08:19:09Z")

</div>

I got the matrices by manually rewriting your function u(x\_1,x\_2) = (x\_1+2)(x\_2+1) to be maximized into the vector form

f(\boldsymbol x) = \frac{1}{2}\boldsymbol x^\top \mathbf Q \boldsymbol x + \mathbf r^\top\boldsymbol x + c.

This conversion of format was nearly instinctive and automatic for me as soon as I observed that it is a quadratic function. Having it then in this compact vector format opens a reservoir of results and tools.

Note that as you can read in the code, \mathbf Q = \begin{bmatrix}0&1\\1&0\end{bmatrix}, which is surely not a null matrix. I also call it Hessian because in this case of a quadratic function this matrix is all that is left after differentiating the function twice with respect to the vector \boldsymbol x (and [Hessian matrix](https://en.wikipedia.org/wiki/Hessian_matrix) is nothing else than a matrix composed of second derivatives).

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 26, 2021, 7:51pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/19 "2021-11-26T19:51:09Z")

</div>

What matrix is Q, I can’t remember the name, but I remember it being the simplest matrix, where each variable is simplified to one.

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [November 26, 2021, 8:00pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/20 "2021-11-26T20:00:48Z")

</div>

No idea what you mean 😀

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 26, 2021, 8:33pm UTC](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043/21 "2021-11-26T20:33:32Z")

</div>

I’m getting an idea of how Jacobians and Hessians work, I think that I’ll need to go over these topics.

One thing I noticed in your plots, and mine when I recreated them, is that the scale is weird. I was able to change the parameters of x1 and x2, but then I added the linear portion, when below zero,and with the ideal marked, it became the same as yours.

[Next page](https://discourse.julialang.org/t/unable-to-use-lagrange-to-solve-2-varable-functions-in-modeling-toolkit/72043.md?page=2)