# Parallel computing using Optim

**URL:** <https://discourse.julialang.org/t/parallel-computing-using-optim/41948>\
**Category:** General Usage\
**Tags:** optim, pmap\
**Created:** [June 23, 2020, 5:28pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948 "2020-06-23T17:28:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [June 23, 2020, 5:28pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/1 "2020-06-23T17:28:44Z")

</div>

Hello,

I’m running the program below on a 32 cpu/64 thread system without much of anything else running on it. If I use anything beyond 16 cores then the execution time in the second run is effectively flat. This is true both when I using a precompiled system image and when I don’t (though a bit more so when using a precompiled system image for reasons I don’t understand). What am I missing?

```julia
using Distributed 
@everywhere using Optim, LinearAlgebra

@everywhere const R = 8000
@everywhere const d = 40

@everywhere function once(x::Int64)
    for r = 1: ((x<0) ? 1 : R)
        function Ω(θ::Vector{Float64})::Float64
            dot(θ,θ) * 0.5
        end
        function dΩ!(g::Vector{Float64}, θ::Vector{Float64})
            g[:] = θ
        end
        function ddΩ!(H::Matrix{Float64}, θ::Vector{Float64})
            H[:,:] .= 0.0
            for i = 1:d
                H[i,i] = 1.0
            end
        end
        Optim.optimize(Ω::Function, dΩ!::Function, ddΩ!::Function, ones(Float64,d), NewtonTrustRegion())
    end
end

function doit()
    @time pmap(once, -64:-1)
    @time pmap(once, 1:192)
end

doit()
```

---

<div class="post-metadata">

**Author:** ![s-broda](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/s-broda/32/3946_2.png) [@s-broda](https://discourse.julialang.org/u/s-broda)\
**Post date:** [June 24, 2020, 8:31am UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/2 "2020-06-24T08:31:29Z")

</div>

Could this be because dot calls BLAS, and Julia is built with OpenBlas limited to use 16 threads, see [https://github.com/JuliaLang/julia/blob/master/deps/blas.mk](https://github.com/JuliaLang/julia/blob/master/deps/blas.mk)?

If this is the case, then you could try writing the dot call as an explicit loop, or using MKL.jl, or building OpenBlas with more threads.

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [June 24, 2020, 1:48pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/3 "2020-06-24T13:48:49Z")

</div>

Thank you. This is indeed related, though the cause is different.

If I run things in parallel and OpenBLAS also runs things in parallel in each of the processes that I have going then I’m effectively using many more threads than I had indicated. Perhaps I should compile a separate version from source with a maximum of one thread to use it like I had intended.

And curiously, my OpenBLAS seems to max out at 8 threads. Hmmm.

---

<div class="post-metadata">

**Author:** ![platawiec](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/platawiec/32/31914_2.png) [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Post date:** [June 24, 2020, 1:59pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/4 "2020-06-24T13:59:23Z")

</div>

This is a bit of speculation, but it could be that threaded BLAS is simply not enabled. See the default keyword argument `enable_threaded_blas = false` for `addprocs`, documentation here: [Distributed Computing · The Julia Language](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.addprocs)

But, because you don’t explicitly call `addprocs`, I’m not sure of the default behavior.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [June 24, 2020, 2:05pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/5 "2020-06-24T14:05:15Z")

</div>

> [@Joris\_Pinkse](#):
>
> Perhaps I should compile a separate version from source with a maximum of one thread to use it like I had intended.

You can control the number of threads used by BLAS at run-time with the `BLAS.set_num_threads` function from `LinearAlgebra`:

```julia
julia> using LinearAlgebra

julia> LinearAlgebra.BLAS.set_num_threads(1)

```

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [June 25, 2020, 3:55pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948/6 "2020-06-25T15:55:49Z")

</div>

Thanks! I didn’t know that was possible.