# Performance of creating model with many terms

**URL:** <https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623>\
**Category:** Optimization (Mathematical)\
**Created:** [November 16, 2021, 11:18pm UTC](https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623 "2021-11-16T23:18:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kiranshila](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kiranshila/32/25475_2.png) [@kiranshila](https://discourse.julialang.org/u/kiranshila)\
**Post date:** [November 16, 2021, 11:18pm UTC](https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623/1 "2021-11-16T23:18:37Z")

</div>

Hey everyone,

I am working on implementing SVMs using JuMP and am running into some performance issues when trying to set up the Dual problem.

Specifically, the setup of the objective function is proving to be very costly. Here is a MWE:

```julia
using JuMP, COSMO, LinearAlgebra

# Kernel function
K = (xₙ, xₘ) -> (1 + xₙ ⋅ xₘ)^2
# Soft marigin
C = 0.01
# Generate a ton of random data
x = rand(7000, 2)
y = rand(7000)

# Setup model
N, d = size(x)
model = Model(COSMO.Optimizer)
@variable(model, 0 <= α[1:N] <= C)
@objective(model,Min,0.5sum(@views α[n]α[m]y[n]y[m]K(x[n,:], x[m,:]) for n ∈ 1:N, m ∈ 1:N) - sum(α))
@constraint(model,α ⋅ y == 0)

```

The `@objective` step is taking on the order of 2 minutes to complete, so any help would be appreciated!

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 16, 2021, 11:47pm UTC](https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623/2 "2021-11-16T23:47:51Z")

</div>

A few things:

- Read the Julia performance tips. No global variables etc.
- Julia is column-major, not row major. Prefer `x[:, n]` over `x[n, :]`
- JuMP prefers explicit summations over linear algebra. (In most cases we can do well, sometimes we hit a slow path. I need to dig in more to understand what’s going on.)
- You have lots of unneeded operations. It looks like your objective is summing over a symmetric matrix. Instead of going over every entry and then multiplying by `0.5`, just do one triangle.

Putting that together, I get:

```nohighlight
using JuMP, LinearAlgebra
function old_foo(N)
    K = (xₙ, xₘ) -> (1 + xₙ ⋅ xₘ)^2
    C = 0.01
    x = rand(N, 2)
    y = rand(N)
    model = Model()
    @variable(model, 0 <= α[1:N] <= C)
    @objective(model,Min,0.5sum(@views α[n]α[m]y[n]y[m]K(x[n,:], x[m,:]) for n ∈ 1:N, m ∈ 1:N) - sum(α))
    @constraint(model,α ⋅ y == 0)
    return model
end

function new_foo(N)
    C = 0.01
    x = rand(2, N)
    y = rand(N)
    function K(n, m)
        tmp = sum(x[i, n] * x[i, m] for i in 1:2)
        return (n == m ? 0.5 : 1) * y[n] * y[m] * (1 + tmp)^2
    end
    model = Model()
    @variable(model, 0 <= α[1:N] <= C)
    @objective(
        model,
        Min, 
        sum(K(n, m) * α[n] * α[m] for n in 1:N, m in 1:n) - 
        sum(α[i] for i in 1:N),
    )
    @constraint(model, sum(α[i] * y[i] for i in 1:N) == 0)
    return model
end

GC.gc()
@time old_foo(2000);
GC.gc()
@time new_foo(2000);

```

( I didn’t want to run the `old_foo(7000)`, it took too long)

```julia
**julia>** GC.gc()

**julia>** @time old_foo(2000);
8.672760 seconds (172.02 M allocations: 13.852 GiB, 17.61% gc time)

**julia>** GC.gc()

**julia>** @time new_foo(2000);
1.948423 seconds (16.02 M allocations: 1.395 GiB, 16.93% gc time)

**julia>** @time new_foo(7000);
30.913255 seconds (196.08 M allocations: 16.975 GiB, 12.20% gc time)

```

---

<div class="post-metadata">

**Author:** ![kiranshila](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kiranshila/32/25475_2.png) [@kiranshila](https://discourse.julialang.org/u/kiranshila)\
**Post date:** [November 16, 2021, 11:49pm UTC](https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623/3 "2021-11-16T23:49:19Z")

</div>

Fantastic! Thank you for your help!

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 17, 2021, 12:40am UTC](https://discourse.julialang.org/t/performance-of-creating-model-with-many-terms/71623/5 "2021-11-17T00:40:29Z")

</div>

Great.

A general rule of thumb is: JuMP is fast, but it’s easy to shoot yourself in the foot. If things are slow, post on the forum and someone will tell you how to improve things.