# Simple Parallel Examples for Embarrassingly Simple Problems

**URL:** <https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788>\
**Category:** Julia at Scale\
**Created:** [February 3, 2018, 2:04am UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788 "2018-02-03T02:04:52Z")\
**Posts on this page:** 10\
**Page:** 2

<div class="post-metadata">

**Author:** ![ivo\_welch](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ivo\_welch](https://discourse.julialang.org/u/ivo_welch)\
**Post date:** [February 5, 2018, 3:34am UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/21 "2018-02-05T03:34:35Z")

</div>

I stand corrected. I can no longer replicate it, either. how strange. it was not a read error. oh well, let’s just ignore. thx, m.

---

<div class="post-metadata">

**Author:** ![ksmcreynolds](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ksmcreynolds/32/860_2.png) [@ksmcreynolds](https://discourse.julialang.org/u/ksmcreynolds)\
**Post date:** [February 5, 2018, 1:32pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/22 "2018-02-05T13:32:41Z")

</div>

> [@ivo\_welch](#):
>
> ksmcreynolds: one strange aspect is that it
> 
> julia\> nprocs()  
> 6
> 
> julia\> rmprocs(3)  
> Task (done) @0x0000000126edb850
> 
> julia\> nprocs()  
> 5
> 
> and further changes do not seem to matter.

`rmprocs(3)` removes processor number 3 not three processors. Before you have processors 1 through 6. After you have processors 1, 2, 4, 5, & 6 which is a total of 5.

---

<div class="post-metadata">

**Author:** ![ivo\_welch](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ivo\_welch](https://discourse.julialang.org/u/ivo_welch)\
**Post date:** [February 5, 2018, 5:07pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/23 "2018-02-05T17:07:08Z")

</div>

My output is below. on my machine, threads have no effect:

| _Method_ | _1 nproc_ | _2 nproc_ | _4 nproc_ | _8 nproc_ | comments |  
| Non Parallel | 0.011 | 0.011 | 0.011 | 0.011 | as expected, constant |  
| Shared Array | 0.014 | 0.011 | 0.008 | 0.008 | good parallelism |  
| Shared Array, Mapreduce | 0.014 | 0.008 | 0.005 | 0.004 | excellent parallelism |  
| Threads | 0.011 | 0.011 | 0.011 | 0.011 | no effect of parallelism |  
| Pmap (Default) | 3.968 | 52.120 | 45.869 | 46.113 | from bad to worse |  
| Pmap (Batch\_Size) | 3.932 | 4.269 | 4.544 | 4.382 | from bad to bad |

and with a longer function

```julia
## const M = 100
## @everywhere function longerfun(x::Int64)::Float64 xs=0.0; for o=1:(500^2); xs+= log(1.0+x); end; sqrt(xs); end#function

```

| _Method_ | _1 nproc_ | _2 nproc_ | _4 nproc_ | _8 nproc_ | comments |  
| Non Parallel | 197 | 197 | 197 | 1971 | as expected, constant |  
| Shared Array | 197 | 102 | 55 | 31 | good parallelism |  
| Shared Array, Mapreduce | 197 | 102 | 55 | 30 | goodparallelism |  
| Threads | 197 | 197 | 197 | 197 | no effect of parallelism |  
| Pmap (Default) | 198 | 112 | 59 | 30 | good parallelism |  
| Pmap (Batch\_Size) | 198 | 104 | 55 | 29 | good parallelism |

which I will also keep at [http://julia.cookbook.tips/doku.php?id=parallel](http://julia.cookbook.tips/doku.php?id=parallel) .

Does anyone know why threads has no parallel features?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 5, 2018, 5:15pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/24 "2018-02-05T17:15:06Z")

</div>

> [@ivo\_welch](#):
>
> My output is below. on my machine, threads have no effect:

Did you set the environment variable?

---

<div class="post-metadata">

**Author:** ![ivo\_welch](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ivo\_welch](https://discourse.julialang.org/u/ivo_welch)\
**Post date:** [February 5, 2018, 7:08pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/25 "2018-02-05T19:08:48Z")

</div>

thanks. obviously not:

```julia
julia> Threads.nthreads()
1

```

I had incorrectly assumed that `-p 8` would increase the number of threads, too, but it did not. duh!  
instead, I had to do `bash> export JULIA_NUM_THREADS=4`.

is there an analog of `julia> addprocs(3)` ? I tried `Threads.addthreads(3)`, but this did not work.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 5, 2018, 7:16pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/26 "2018-02-05T19:16:05Z")

</div>

Threads have to be setup before the application is opened (at least for Julia), so I don’t think that’s possible.

---

<div class="post-metadata">

**Author:** ![t-pollington](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/t-pollington/32/20074_2.png) [@t-pollington](https://discourse.julialang.org/u/t-pollington)\
**Post date:** [April 16, 2021, 2:56pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/27 "2021-04-16T14:56:58Z")

</div>

I’ve found it pretty tricky to find a pseudocode example for sampling Monte Carlo simulations from a simulation function. Despite this topic being 3 years old, it’s still high on Google search results, hence the explanation here (abridged from @cecileane: [notes](http://cecileane.github.io/computingtools/pages/notes1209.html)).

**Newbie tips**

- @parallel was deprecated since _Julia_ v0.7 and replaced with @distributed in Distributed package.
- @distributed OK for approx equal workload split per workers. Else consider `pmap` for load balancing and keeping idle workers busy!

**Purpose** : Minimal working example for independent `nsims` (embarassingly parallel _e.g._ Monte Carlo simulations) of roughly equal work or when `nsims`\<available workers.

```julia
using Distributed
addprocs(2) # say

@everywhere begin
	using Distributed
	using SharedArrays
end

# define write vars beforehand
rwvar1 = SharedArray{Float64}(nsims)
[...]

# define read vars as normal
rvar1 = Array{Float64}(nsims)
[...]

# define simulationFunction as normal but made available to all workers
@everywhere function simulationFunction(rwvar1,...,rvar1,...)
	[...]
	return rwvar1[i] ...
end

# run it in parallel
@sync @distributed for i in 1:nsims
  out = simulationFunction(rwvar1[i],...,rvar1[i],...)
  rwvar1[i], ... = out
end

# save rwvar1,... as normal

```

To access the data in a `SharedArray` type, `rwvar1.s` is needed which then just returns a normal `Array` type which I found was easier to then save as a `.jld` for loading in _R_ later.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 16, 2021, 3:10pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/28 "2021-04-16T15:10:46Z")

</div>

Any idea what algorithm `JuliaLibm.log1p` uses? `log1p` is pretty important, so if `Base` is under-performing, I might want to take a look.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 23, 2021, 2:12pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/29 "2021-04-23T14:12:51Z")

</div>

AFAIK this was fixed by

[https://github.com/JuliaLang/julia/pull/24750](https://github.com/JuliaLang/julia/pull/24750)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 23, 2021, 3:45pm UTC](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788/30 "2021-04-23T15:45:18Z")

</div>

Ah. I looked at that PR, but I did a git blame and saw that the code was from 2017, so I thought it couldn’t be the cause, and never realized that it was only merged in 2018.

[Previous page](https://discourse.julialang.org/t/simple-parallel-examples-for-embarrassingly-simple-problems/8788.md?page=1)