# Write large Least Square like problems in JuMP

**URL:** <https://discourse.julialang.org/t/write-large-least-square-like-problems-in-jump/35931>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [March 13, 2020, 3:52pm UTC](https://discourse.julialang.org/t/write-large-least-square-like-problems-in-jump/35931 "2020-03-13T15:52:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmetivie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmetivie/32/6926_2.png) [@dmetivie](https://discourse.julialang.org/u/dmetivie)\
**Post date:** [March 13, 2020, 3:52pm UTC](https://discourse.julialang.org/t/write-large-least-square-like-problems-in-jump/35931/1 "2020-03-13T15:52:18Z")

</div>

Hi,  
Here is a small code showing the issue I am facing when trying to form large problem (least square like) with JuMP

```julia
using JuMP, Ipopt, BenchmarkTools

p = 500
n = 200

solver = with_optimizer(Ipopt.Optimizer);
ML = Model(solver)
@variable(ML, v[1:p] )

A = rand(n,p)
b = [ones(15); zeros(p-15)];
y = A*b+0.5*rand(n);

@btime sum( (A*v - y).^2 ); # Computing sum of square where v are JuMP variables
30.526 s (50026082 allocations: 2.55 GiB)
@btime sum( (A*rand(500) - y).^2 ); # Same sum when v are floats 
33.332 μs (10 allocations: 9.47 KiB)

```

The problem is not so big and it still takes 30s to form (I am not solving it yet).  
It is normal that the second sum is faster but here it is `10^6` times faster!  
My questions are:

- Is there an efficient way to form that type of problem  
`@objective(ML, Min, sum( (A*v - y).^2 ) )`?  
Is it normal that simple square sum is so long to form with JuMP?  
I want to deal with `p ~ 5000`, `n ~ 1000` plus some handmade regularization terms and as for now JuMP is not an option `~3000s`.

- Without JuMP, are there efficient packages that one could use? `Convex.jl` seems faster but I am not sure if I can add custom (non-convex) regularization terms.

---

<div class="post-metadata">

**Author:** ![miles.lubin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/miles.lubin/32/279_2.png) [@miles.lubin](https://discourse.julialang.org/u/miles.lubin)\
**Post date:** [March 13, 2020, 4:13pm UTC](https://discourse.julialang.org/t/write-large-least-square-like-problems-in-jump/35931/2 "2020-03-13T16:13:53Z")

</div>

> [@dmetivie](#):
>
> Is there an efficient way to form that type of problem  
> `@objective(ML, Min, sum( (A*v - y).^2 ) )` ?

Yes, define a new set of variables `z`, add the constraint `z == A*v - y`, and minimize `sum(z.^2)`. To see why this helps, compare the density of the Hessian matrices of the two formulations.

Ipopt is really designed for constrained nonlinear optimization. If you only have a least squares problem plus other objective terms, you can use other packages like [Optim](https://github.com/JuliaNLSolvers/Optim.jl). Others will likely chime in with additional alternatives.

---

<div class="post-metadata">

**Author:** ![dmetivie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmetivie/32/6926_2.png) [@dmetivie](https://discourse.julialang.org/u/dmetivie)\
**Post date:** [March 16, 2020, 5:43pm UTC](https://discourse.julialang.org/t/write-large-least-square-like-problems-in-jump/35931/3 "2020-03-16T17:43:08Z")

</div>

Thanks! It works well and way faster

```julia
@variable(ML, z[1:n] )
btime @constraint(ML, z .== A*v - y)
82.711 ms (116252 allocations: 14.70 MiB)
@btime @objective(ML, Min, sum( z.^2 ) )
 231.922 μs (2854 allocations: 238.06 KiB)

```