# Why isn't pmap faster than map in this example?

**URL:** <https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091>\
**Category:** Performance\
**Created:** [July 1, 2018, 5:44pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091 "2018-07-01T17:44:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [July 1, 2018, 5:44pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/1 "2018-07-01T17:44:33Z")

</div>

I have a function that computes the mean of each column of a matrix 100 times. However, the serial version is as fast as the parallel version on my machine (3.5 GHz Intel Core i5 with 4 cores)

```julia
addprocs(3)
@everywhere function repeated_mean(x)
   for _ in 1:100
      mean(x)
   end
   return x
end
function f(X)
   @time map(repeated_mean, X)
   @time pmap(repeated_mean, X)
   return nothing
end
f([rand(10_000_000) for j in 1:20])
#> 8.432926 seconds (2 allocations: 256 bytes)
#> 9.528860 seconds (377.27 k allocations: 1.524 GiB, 8.65% gc time)

```

What is happening? I naively expected the parallel version to be 4 times faster. Is there a better way to exploit multicores in this kind of computation?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 1, 2018, 5:51pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/2 "2018-07-01T17:51:02Z")

</div>

I would try using a `tmap`

[https://github.com/bkamins/KissThreading.jl](https://github.com/bkamins/KissThreading.jl)

Make sure you enable threads of course.

---

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [July 1, 2018, 5:59pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/3 "2018-07-01T17:59:40Z")

</div>

@ChrisRackauckas using multithreading is a bit faster, but only by 6%

```julia
using KissThreading
function f(X)
   @time map!(repeated_mean, similar(X), X)
   @time tmap!(repeated_mean, similar(X), X)
   return nothing
end
Threads.nthreads()
#> 4
f([rand(10_000_000) for j in 1:20])
#> 8.242080 seconds (1 allocation: 240 bytes)
#> 7.594405 seconds (154 allocations: 10.547 KiB)

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 1, 2018, 6:08pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/4 "2018-07-01T18:08:41Z")

</div>

The mean calculation is probably already highly parallel due to SIMD and IO-bound instead of compute bound since the computation is really cheap. Try a more expensive calculation.

---

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [July 1, 2018, 6:40pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/5 "2018-07-01T18:40:52Z")

</div>

I see. That’s the kind of simple computation I am interested in though. Does SIMD really use multicores?

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [July 1, 2018, 6:48pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/6 "2018-07-01T18:48:11Z")

</div>

SIMD happens at each core. If your problem is memory or latency-bound, your best option is closely examine the problem to see if there are opportunities to cut down on memory access.

---

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [July 1, 2018, 6:53pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/7 "2018-07-01T18:53:53Z")

</div>

Just to be clear (sorry I don’t know anything about this), IO-bound means that, even though each core reads a different vector, parallelization is slowed down because, in some vague sense, all the cores use a common stuff to read a vector? Is it related to having multiple cpus with one socket vs multiple cpus with multiple sockets?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 1, 2018, 7:28pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/8 "2018-07-01T19:28:56Z")

</div>

> [@matthieu](#):
>
> IO-bound means that, even though each core reads a different vector, parallelization is slowed down because, in some vague sense, all the cores use a common stuff to read a vector?

It means that the computation is bound by the speed of moving things into and out of use/cache. Something like matrix multiplication re-uses the values multiple times. Other computations like exponents are just expensive. A bunch of additions in a row where each value is only used once can only be parallelized so much until it just can’t feed values in fast enough. As computations get more expensive, they are less bound by IO and that’s where parallelization is more useful (well, you can parallelize in IO bound cases, but not well locally like this).

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [July 1, 2018, 7:36pm UTC](https://discourse.julialang.org/t/why-isnt-pmap-faster-than-map-in-this-example/12091/9 "2018-07-01T19:36:51Z")

</div>

An illustrative example (on a dual-core i5):

```julia
using BenchmarkTools
addprocs(Sys.CPU_CORES)

@everywhere function repeated_mean(x)
   s = zero(eltype(x))
   for i = 1:100
      s += mean(x)
   end
   return s
end

@everywhere function repeated_hard_mean(x)
   s = zero(eltype(x))
   for i = 1:100
      s += mean(log(sin(exp(xi))) for xi in x)
   end
   return s
end

```

```julia
X = [rand(100_000) for j in 1:20]

@btime map(repeated_mean, $X) # 30.8 ms
@btime pmap(repeated_mean, $X) # 28.3 ms

@btime map(repeated_hard_mean, $X) # 11.3 s
@btime pmap(repeated_hard_mean, $X) # 3.61 s

```