# Parallelization Optimization Runs

**URL:** <https://discourse.julialang.org/t/parallelization-optimization-runs/100667>\
**Category:** Optimization (Mathematical)\
**Created:** [June 21, 2023, 5:30pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667 "2023-06-21T17:30:54Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![A\_M](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/a_m/32/46038_2.png) [@A\_M](https://discourse.julialang.org/u/A_M)\
**Post date:** [June 21, 2023, 5:30pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/1 "2023-06-21T17:30:54Z")

</div>

I need to run over a million small optimization problems which are similar from the perspective of the structure but have different coefficients for constraint and objective. I know I can create and solve many models concurrently, but I am not sure that is the most efficient way.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 21, 2023, 5:32pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/2 "2023-06-21T17:32:46Z")

</div>

What kind of optimization problems?

---

<div class="post-metadata">

**Author:** ![A\_M](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/a_m/32/46038_2.png) [@A\_M](https://discourse.julialang.org/u/A_M)\
**Post date:** [June 21, 2023, 6:42pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/4 "2023-06-21T18:42:56Z")

</div>

For example, this code works but is inefficient since building a model from scratch.

```julia
using JuMP, GLPK, Random

function generate_model(seed)
    Random.seed!(seed)
    model = Model(GLPK.Optimizer)
    @variable(model, 0 <= x <= 2)
    @variable(model, 0 <= y <= 30)
    @constraint(model, x + y <= 20)
    @constraint(model, y >= 10)
    @objective(model, Max, (5*rand())*x + (3*rand())*y)
    return model
end

function solve_model(seed)
    model = generate_model(seed)
    optimize!(model)
    if termination_status(model) == MOI.OPTIMAL
        return value.(all_variables(model))
    else
        error("Problem could not be solved to optimality")
    end
end

seeds = 1:10 # define seeds

tasks = [Threads.@spawn solve_model(seed) for seed in seeds]

solutions = fetch.(tasks)

for (i, sol) in enumerate(solutions)
    println("Solution to model ", i, ": ", sol)
end

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 21, 2023, 6:52pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/5 "2023-06-21T18:52:50Z")

</div>

Create one model per task, and update the seed within the task, in a loop inner to the task.

---

<div class="post-metadata">

**Author:** ![A\_M](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/a_m/32/46038_2.png) [@A\_M](https://discourse.julialang.org/u/A_M)\
**Post date:** [June 21, 2023, 7:50pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/6 "2023-06-21T19:50:32Z")

</div>

Isn’t what I did in my previous example?

The idea is how to create an incremental interface and use copy\_to.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 21, 2023, 8:11pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/7 "2023-06-21T20:11:47Z")

</div>

There an interface of JuMP that allows updating the model, without having to create a new one everytime when parameters change. Using it you can create one model per task, and within each task run a loop that solves the problem many times, updating the model locally. I’m not at the computer neither know by heart that interface, but It exists.

(I mean one model per thread, if tou want. What I mean is only spawing a number of tasks much smaller than the number of optimizatios)

---

<div class="post-metadata">

**Author:** ![A\_M](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/a_m/32/46038_2.png) [@A\_M](https://discourse.julialang.org/u/A_M)\
**Post date:** [June 21, 2023, 8:26pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/8 "2023-06-21T20:26:47Z")

</div>

OK, I tried many approaches, including the incremental interface, which didn’t work with concurrency. But that would be great if you provide an example that does what you said.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 21, 2023, 8:36pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/9 "2023-06-21T20:36:13Z")

</div>

Maybe here you can find useful info: [How to optimize multiple times in a run - #2 by odow](https://discourse.julialang.org/t/how-to-optimize-multiple-times-in-a-run/75118/2)

More specifically : [Power Systems · JuMP](https://jump.dev/JuMP.jl/stable/tutorials/applications/power_systems/#Modifying-the-JuMP-model-in-place)

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 21, 2023, 8:39pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/10 "2023-06-21T20:39:55Z")

</div>

See [Parallelism · JuMP](https://jump.dev/JuMP.jl/stable/tutorials/algorithms/parallelism/)

Here’s an example:

```julia
using JuMP, HiGHS, Random

function solve_seeds(seeds::AbstractVector{Int})
    model = Model(HiGHS.Optimizer)
    set_silent(model)
    @variable(model, 0 <= x <= 2)
    @variable(model, 0 <= y <= 30)
    @constraint(model, x + y <= 20)
    @constraint(model, y >= 10)
    solutions = Vector{Float64}[]
    for seed in seeds
        Random.seed!(seed)
        @objective(model, Max, (5*rand())*x + (3*rand())*y)
        optimize!(model)
        if termination_status(model) == MOI.OPTIMAL
            push!(solutions, value.(all_variables(model)))
        else
            error("Problem could not be solved to optimality")
        end
    end
    return solutions
end

function partition_array(x::AbstractVector, N::Int)
    k = ceil(Int, length(x) / N)
    return [x[(k*(i-1)+1):min(length(x), k * i)] for i in 1:N]
end

seeds = 1:10 # define seeds
solution_dict = Dict{Int,Vector{Float64}}()
my_lock = Threads.ReentrantLock()
Threads.@threads for partition in partition_array(seeds, Threads.nthreads())
    solutions = solve_seeds(partition)
    Threads.lock(my_lock) do 
        for (seed, solution) in zip(partition, solutions)
            solution_dict[seed] = solution
        end
    end
end
solution_dict

```

---

<div class="post-metadata">

**Author:** ![A\_M](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/a_m/32/46038_2.png) [@A\_M](https://discourse.julialang.org/u/A_M)\
**Post date:** [June 21, 2023, 9:30pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/11 "2023-06-21T21:30:18Z")

</div>

Thanks @odow

1. I believe a channel is more efficient than a lock based on my experience coding golang. I guess similar applies to Julia. You can put the solutions in the Channel when all threads are done and copy them over instead of locking the whole array.

2. you are adding a new objective instead of updating it. That can be doable, but imagine when you need to update all the constraints and the objective, then building it from scratch can be easier, right?

3. I was hoping for a method that creates a deep copy of a model and then updates the copied model coefficients.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [June 21, 2023, 9:48pm UTC](https://discourse.julialang.org/t/parallelization-optimization-runs/100667/12 "2023-06-21T21:48:52Z")

</div>

1. This is almost certainly not a bottleneck. Don’t worry about the performance. Solving the LP is _much_ more expensive.
2. There are methods you can use to modify individual coefficients in various places:
  - [set\_objective\_coefficient](https://jump.dev/JuMP.jl/dev/api/JuMP/#set_objective_coefficient)
  - [set\_normalized\_coefficient](https://jump.dev/JuMP.jl/dev/api/JuMP/#set_normalized_coefficient)
  - [set\_normalized\_rhs](https://jump.dev/JuMP.jl/dev/api/JuMP/#set_normalized_rhs)  
See also
  - [Constraints · JuMP](https://jump.dev/JuMP.jl/dev/manual/constraints/#Modify-a-constant-term)
  - [Constraints · JuMP](https://jump.dev/JuMP.jl/dev/manual/constraints/#Modify-a-variable-coefficient)
  - [Objectives · JuMP](https://jump.dev/JuMP.jl/dev/manual/objective/#Modify-an-objective-coefficient)  
There’s a trade-off, where if you changing almost every coefficient, then sure, it can be more efficient to rebuild from scratch.

3. I don’t know if I understand the suggestion. In general, I’d recommend against copying a JuMP model. There are lots of subtle things that can go wrong.