# Parallel processing

**URL:** <https://discourse.julialang.org/t/parallel-processing/37319>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [April 10, 2020, 8:30am UTC](https://discourse.julialang.org/t/parallel-processing/37319 "2020-04-10T08:30:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ohmsweetohm1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ohmsweetohm1/32/49126_2.png) [@ohmsweetohm1](https://discourse.julialang.org/u/ohmsweetohm1)\
**Post date:** [April 10, 2020, 8:30am UTC](https://discourse.julialang.org/t/parallel-processing/37319/1 "2020-04-10T08:30:17Z")

</div>

I did some experiments with parallel processing, i.e. solving models in parallel. But the result is not what I expected:

```julia
using Distributed
import JuMP

addprocs(4)

function simple_model(a, b)
    model = JuMP.Model()

    JuMP.@variable(model, 0 <= x <= 2)
    JuMP.@variable(model, 0 <= y <= 30)

    JuMP.@objective(model, Max, a*x + b*y)
    JuMP.@constraint(model, 1x + 5y <= 3.0)
    return model
end

@everywhere import Clp, JuMP
@everywhere function solve_model(model)
    optimizer = Clp.Optimizer
    optimizer_params = Dict("LogLevel" => 0)
    JuMP.set_optimizer.(model, optimizer)
    JuMP.set_optimizer_attributes.(model, optimizer_params...)
    JuMP.optimize!(model)
    return
end

models = [simple_model(rand(), rand()) for i in 1:1000];

t0 = time()
for i in 1:10
    map(solve_model, models)
end
t1 = time()
println("Serial execution: elapsed time: ", (t1-t0)/10, " seconds")

t0 = time()
for i in 1:10
    pmap(solve_model, models, batch_size=1)
end
t1 = time()
println("Parallel execution: elapsed time: ", (t1-t0)/10 " seconds")

```

> Serial execution: elapsed time: 1.6002999782562255 seconds  
> Parallel execution: elapsed time: 3.769099998474121 seconds

If I increase the `batch_size` to 10 I get better results:

> Serial execution: elapsed time: 1.470799994468689 seconds  
> Parallel execution: elapsed time: 1.8813000202178956 seconds

But still, the parallel processing approach is slower.

Am I doing something wrong?

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [April 10, 2020, 11:03am UTC](https://discourse.julialang.org/t/parallel-processing/37319/2 "2020-04-10T11:03:14Z")

</div>

> [@ohmsweetohm1](#):
>
> JuMP.@variable(model, 0 \<= x \<= 2)

What are the specifications of the server or laptop you are running this on?  
How many cores? Adn is hyperthreading enabled?

---

<div class="post-metadata">

**Author:** ![ohmsweetohm1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ohmsweetohm1/32/49126_2.png) [@ohmsweetohm1](https://discourse.julialang.org/u/ohmsweetohm1)\
**Post date:** [April 10, 2020, 1:28pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/3 "2020-04-10T13:28:14Z")

</div>

4 cores, Core i5, 8GB RAM, yes it is enabled.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 10, 2020, 1:30pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/4 "2020-04-10T13:30:48Z")

</div>

A few things.

- Parallel processing has an overhead. So you will probably only see a speed up on bigger models/number of models
- One issue is that you’re having to copy the model to each proc. Try just passing `a` and `b`, and building and solving each model on the remote procs.
- This won’t help with Clp for some internal reasons, but if you use a solver like GLPK or CPLEX, you can build the model once on each worker, and then just call `@objective` and solve for each new `a` and `b`. Then you’ll only pay the build cost once.

---

<div class="post-metadata">

**Author:** ![ohmsweetohm1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ohmsweetohm1/32/49126_2.png) [@ohmsweetohm1](https://discourse.julialang.org/u/ohmsweetohm1)\
**Post date:** [April 10, 2020, 1:33pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/5 "2020-04-10T13:33:21Z")

</div>

Thanks!  
This was just an example. Only changing the objective is not enough in my application.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 10, 2020, 1:37pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/6 "2020-04-10T13:37:48Z")

</div>

Okay, then that simplifies things, minimize the data-movement, and pass the data necessary to build the model, not the built JuMP model.

```nohighlight
data = [(1, 1), (2, 3)]
@everywhere function build_and_solve(data)
    return data[1] + data[2]
end

pmap(build_and_solve, data)

```

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [April 10, 2020, 2:54pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/7 "2020-04-10T14:54:40Z")

</div>

Also remember with hyperthreading you will have only 2 real cores.  
When you have followed @odow excellent advice then I would run with one proc, two procs and 4 procs

---

<div class="post-metadata">

**Author:** ![ccoffrin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ccoffrin/32/400_2.png) [@ccoffrin](https://discourse.julialang.org/u/ccoffrin)\
**Post date:** [April 10, 2020, 3:19pm UTC](https://discourse.julialang.org/t/parallel-processing/37319/8 "2020-04-10T15:19:02Z")

</div>

FWIW, I have had good success using `Distributed` / `pmap` and the approach that @odow suggests for setting up and solving the models independently in each worker.

In my case the JuMP model solves were on the order of 10-30 seconds, which was sufficient to outweigh the parallelization overheads.