# Struggling with pmap

**URL:** <https://discourse.julialang.org/t/struggling-with-pmap/28432>\
**Category:** New to Julia\
**Tags:** parallel\
**Created:** [September 5, 2019, 2:55pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432 "2019-09-05T14:55:01Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [September 5, 2019, 2:55pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/1 "2019-09-05T14:55:01Z")

</div>

Hello,

I am struggling to understand why `pmap` is slow. I use the following fast function `rand()`.

```julia
using Distributed
addprocs(4)
@everywhere using Random
@everywhere g(x) = rand()
 @time map(g,1:100_000_000);
  0.697132 seconds (10 allocations: 762.940 MiB, 3.11% gc time)
@time pmap(g,1:100_000_000);
# still running...

```

I my use case, I have a function `g` which performs stochastic simulations and is quite fast to execute. I want to run it million of times though.

I am sorry if this is a trivial question, but can someone give me a hint about this behaviour and possibly how to improve it.

Thank you

Best regards

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [September 5, 2019, 2:58pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/2 "2019-09-05T14:58:43Z")

</div>

Can you communicate with those distributed processes? Are they running on your local machine? I guess so…  
Is it possible that you have used a lot of memory and have started swapping on your local machine?

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [September 5, 2019, 3:10pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/3 "2019-09-05T15:10:19Z")

</div>

> [@johnh](#):
>
> Can you communicate with those distributed processes? Are they running on your local machine? I guess so

You are right. Single machine multiprocessors.

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [September 5, 2019, 3:24pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/4 "2019-09-05T15:24:39Z")

</div>

It may have to do with the overhead of `pmap`. I’ve heard in general that `pmap` should be used when the function `g` does a large amount of work to offset the cost of overhead. In your case, apparently, the function `g` is much faster than the time it takes to send out the work to different processors. If you want to parallelize a fast function (such as your `g`), either use low level primitives such as `spawnat` and `fetch` or macros like `@parallel` on your for loop.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [September 5, 2019, 3:43pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/5 "2019-09-05T15:43:18Z")

</div>

What do these retuen after you addprocs ?  
nprocs()  
workers()

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [September 5, 2019, 3:44pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/6 "2019-09-05T15:44:58Z")

</div>

@affans makes an excellent point. If you are going to parallelize by distrributing functions you need to give each worker a ‘decent’ amount of work to do. May apology to non-English native speakers.  
As @affans says the ration of the time taken to do the task should be greater than the time to comminucate, send out and return the data.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [September 5, 2019, 3:59pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/7 "2019-09-05T15:59:48Z")

</div>

Using your suggestions, I got it to work better (not for the MWE posted here) by making `g` computing more. Something along those lines:

```julia
@everywhere g(x) = rand(1000_000)
@time pmap(g,1:100);

```

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [September 5, 2019, 4:10pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/8 "2019-09-05T16:10:30Z")

</div>

I still think that `pmap` is not the way to go here (in the context of your function). See this excerpt from the documentation:

> Julia’s [`pmap`](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.pmap) is designed for the case where each function call does a large amount of work. In contrast, `@distributed for` can handle situations where each iteration is tiny, perhaps merely summing two numbers. Only worker processes are used by both [`pmap`](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.pmap) and `@distributed for` for the parallel computation. In case of `@distributed for` , the final reduction is done on the calling process.

Can you try `@distributed for` and see if it speeds up your result?

---

<div class="post-metadata">

**Author:** ![raminammour](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raminammour/32/13572_2.png) [@raminammour](https://discourse.julialang.org/u/raminammour)\
**Post date:** [September 5, 2019, 4:45pm UTC](https://discourse.julialang.org/t/struggling-with-pmap/28432/9 "2019-09-05T16:45:58Z")

</div>

This thread may help too, it echos what was said above:

> [@Weird behavior of pmap](https://discourse.julialang.org/t/weird-behavior-of-pmap/25910):
>
> The following suboptimal behavior of the parallel map has been observed: 53.927 s (82132566 allocations: 2.81 GiB) 2.011 s (8000041 allocations: 167.85 MiB) 18.922 ms (2 allocations: 7.63 MiB) 20.058 ms (3 allocations: 7.63 MiB) The above was obtained with N = 1\_000\_000 a = rand(N) + rand(N)\*1im using BenchmarkTools using Distributed addprocs(2) @btime aa = pmap(x -\> abs(x), $a) rmprocs(workers()) @btime aa = pmap(x -\> abs(x), $a) @btime aa = abs.($a) @btime aa = map(x -\> abs(x), $a…