# Multithreading and pmap **URL:** \ **Category:** Julia at Scale\ **Created:** [January 4, 2019, 8:29pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284 "2019-01-04T20:29:43Z")\ **Posts on this page:** 9\ **Page:** 1

**Author:** ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)\ **Post date:** [January 4, 2019, 8:29pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/1 "2019-01-04T20:29:43Z")

I have access to a new machine that has 2 x 20 core Xeon Gold CPUs which support multithreading. I was playing around with some simple `pmap` problems, and I was not seeing a performance jump when I tried going from 40 to 80 workers (hoping to benefit from the multithreading). Does anyone have any suggestions on how to best leverage my computing environment with Julia for embarassingly parallel (i.e., `pmap`) type problems? ---

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\ **Post date:** [January 4, 2019, 9:02pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/2 "2019-01-04T21:02:15Z")

Have you set the environment variable `JULIA_NUM_THREADS` or started julia with `julia -p 40` to start julia with 40 worker processes? ---

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\ **Post date:** [January 4, 2019, 9:30pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/3 "2019-01-04T21:30:51Z")

`JULIA_NUM_THREADS` doesn’t apply to `pmap`; that comes into play with `Threads.@threads for`. @gideonsimpson, why would you expect performance to improve if you go from 40 to 80 workers if you only have 40 logical cores? ---

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\ **Post date:** [January 4, 2019, 9:34pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/4 "2019-01-04T21:34:27Z")

> [@tkoolen](#): > > `JULIA_NUM_THREADS` doesn’t apply to `pmap` I’m aware - but it may be that the code from @gideonsimpson uses that macro in some place and they expected an increase in performance. I don’t know, as they didn’t specify in the first post 🙂 ---

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\ **Post date:** [January 4, 2019, 10:27pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/5 "2019-01-04T22:27:51Z")

I think he meant that he has 40 physical cores which support _hyper_threading, ie. 80 threads. ---

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\ **Post date:** [January 4, 2019, 10:40pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/6 "2019-01-04T22:40:16Z")

Even if that’s the case, I actually wouldn’t expect performance to improve if you increase the number of workers past the number of physical cores. ---

**Author:** ![gideonsimpson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gideonsimpson/32/1928_2.png) [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)\ **Post date:** [January 4, 2019, 10:48pm UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/7 "2019-01-04T22:48:19Z")

I use `addprocs` at the beginning of my script, which I call with just `julia script1.jl` ---

**Author:** ![kolia](https://avatars.discourse-cdn.com/v4/letter/k/c68b51/32.png) [@kolia](https://discourse.julialang.org/u/kolia)\ **Post date:** [January 5, 2019, 12:50am UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/8 "2019-01-05T00:50:03Z")

To be clear, pmap distributes computation across processes which do not share memory; objects get serialized to get sent between processes, and the processes can run on remote machines. Maybe things aren’t speeding up when you add processes because of this serialization overhead? Multi-threading on the other hand is shared-memory so does not incur that overhead. It is most easily used with the @threads macro, not pmap, and all threads are on the same machine. Multi-threading is experimental, but mostly works unless you’re doing IO on the threads. So you could try the @threads macro instead, but then you’d need to launch Julia in an environment where JULIA\_NUM\_THREADS is set for julia’s Threads to run on multiple cpu cores, otherwise all your @threads will do is run a bunch of tasks on the same core. ---

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\ **Post date:** [January 5, 2019, 1:50am UTC](https://discourse.julialang.org/t/multithreading-and-pmap/19284/9 "2019-01-05T01:50:59Z")

> [@kolia](#): > > o you could try the @threads macro instead, but then you’d need to launch Julia in an environment where JULIA\_NUM\_THREADS is set for julia’s Threads to run on multiple cpu cores, otherwise all your @threads will do is run a bunch of tasks on the same core. If he goes the threading route, I’d recommend taking a look at [KissThreading.jl](https://github.com/bkamins/KissThreading.jl). Among other things, it offers a `tmap!` function and initializes a vector of Mersenne Twisters named `TRNG` that you can use if any of the code generates random numbers. `pmap` and `tmap!` are better than `@distributed for` and `@threads for` when the functions being called take a while, and there’s some variance in that run time. The former use dynamic scheduling, and the latter static scheduling. I’ve normally tried using threads before distributed. However, I normally get poor scaling. Much worse than OpenMP. It’s probably my fault.