# JuliaOpt parallel algorithms
**URL:** https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093
**Category:** Optimization (Mathematical)
**Tags:** first-steps
**Created:** [July 27, 2017, 2:49pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093 "2017-07-27T14:49:12Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@ivborissov](https://discourse.julialang.org/u/ivborissov)
#### Post date: [July 27, 2017, 2:49pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/1 "2017-07-27T14:49:12Z")
Hi,
I am currently using NLopt.jl to estimate parameters of differential equation systems. I am interested to speed up the calculations hence I am searching for solutions to solve optimization problem in parallel (using multiple workers on different nodes, threads, etc). Are there ways to handle it using JuliaOpt?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 27, 2017, 2:56pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/2 "2017-07-27T14:56:59Z")
An important point to add is that @ivborissov’s objective function cannot easily be parallelized over more than 3 points, meaning that it needs to be parallelized elsewere (usually this is the place to parallelize!).
One trivial way to parallelize optimization problems is to use local optimizers and parallelize the search domain.
```julia
all_minx = [Float64[] for i in Threads.nthreds()]
all_minf = [Float64[] for i in Threads.nthreds()]
Threads.@threads for i in 1:N
(minf,minx,ret) = optimize(opt, rand(2))
push!(all_minx[Threads.threadid()],minx)
push!(all_minf[Threads.threadid()],minf)
end
minxs = vcat(all_minx...)
minfs = vcat(all_minf...)
```
That’s an easy way to solve with `N` random initial conditions in parallel using threads, and collect the `minx` and `minf` values. You can then extend this to solve using a grid of initial conditions by using `i` to determine what the IC is, or you can change this to be an `@parallel` or `pmap` call. This is then useful because local optimizers converge to local minimums, so if you have a good enough coverage you’ll hit the global one, so the “true minimum” can be taken as `minimum(minfs)` (and that index in `minxs` gives the minarg).
This is different from a parallel optimization algorithm though, and I am not of an implementation which exists.
---
### Author:  [@ivborissov](https://discourse.julialang.org/u/ivborissov)
#### Post date: [July 31, 2017, 8:31am UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/3 "2017-07-31T08:31:36Z")
Chris,thanks! Yes it is a good idea to solve the problem with a wide range of initial conditions. Also I would like to know if there are optimization algorithms that can be effectivly parallelized?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 31, 2017, 9:03am UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/4 "2017-07-31T09:03:28Z")
> [@ivborissov](#):
>
> Also I would like to know if there are optimization algorithms that can be effectivly parallelized?
Most of the standard methods are very sequential and so they cannot. If the system is large enough, then it can be pretty effectively parallelized by parallelizing the solution of linear systems and having the user parallelize the objective function. Since many optimization algorithms have to do some form of linear solving as the main step, this then works for sufficiently large systems.
But form what I know about your system from Gitter chats, it doesn’t fall into this range. In this case, evolutionary and particle swarm based methods are the methods I know which can be parallelized. Essentially these work by having a ton of different individual agents moving through parameter space, and so you can parallelize across the agents quite effectively. I don’t think these algorithms are implemented in Julia, or at least I haven’t seen a package for it. But you can take a look at the Simulated Annealing implementation of Optim.jl (though be careful: [Experience with SimulatedAnnealing? · Issue #173 · JuliaNLSolvers/Optim.jl · GitHub](https://github.com/JuliaNLSolvers/Optim.jl/issues/173)) or take a look at Evolutionary.jl and maybe build (/contribute?) a parallel algorithm from one of those.
P.S. There is this package but I’ve never used it.
> **[GitHub - floswald/SMM.jl: Simulated Method of Moments for Julia](https://github.com/floswald/SMM.jl)**
>
> Simulated Method of Moments for Julia. Contribute to floswald/SMM.jl development by creating an account on GitHub.
---
### Author:  [@ivborissov](https://discourse.julialang.org/u/ivborissov)
#### Post date: [July 31, 2017, 2:02pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/5 "2017-07-31T14:02:00Z")
Chris, thanks.
There are some really nice ideas how to run evolutionary algorithms in parallel
> **[parallel-eas.pdf](https://staffwww.dcs.shef.ac.uk/people/D.Sudholt/parallel-eas.pdf)**
>
> 414.95 KB
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [July 31, 2017, 5:25pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/6 "2017-07-31T17:25:44Z")
Some parallel evolutionary algorithms are available in the package [https://github.com/robertfeldt/BlackBoxOptim.jl/blob/master/README.md](https://github.com/robertfeldt/BlackBoxOptim.jl/blob/master/README.md)
Hopefully simple to test and see if they work for you, the Readme links to a parallel example
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 31, 2017, 5:51pm UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/7 "2017-07-31T17:51:58Z")
> [@baggepinnen](#):
>
> Some parallel evolutionary algorithms are available in the package
Oh nice! Never knew BlackBoxOptim.jl had this!
---
### Author:  [@ivborissov](https://discourse.julialang.org/u/ivborissov)
#### Post date: [August 1, 2017, 8:43am UTC](https://discourse.julialang.org/t/juliaopt-parallel-algorithms/5093/8 "2017-08-01T08:43:38Z")
Great, thanks! I will test if it works for my problem