Simple Parallel Examples for Embarrassingly Simple Problems

Specific Domains Julia at Scale
21

I stand corrected. I can no longer replicate it, either. how strange. it was not a read error. oh well, let’s just ignore. thx, m.

22

rmprocs(3) removes processor number 3 not three processors. Before you have processors 1 through 6. After you have processors 1, 2, 4, 5, & 6 which is a total of 5.

1 Like
23

My output is below. on my machine, threads have no effect:

| Method | 1 nproc | 2 nproc | 4 nproc | 8 nproc | comments |
| Non Parallel | 0.011 | 0.011 | 0.011 | 0.011 | as expected, constant |
| Shared Array | 0.014 | 0.011 | 0.008 | 0.008 | good parallelism |
| Shared Array, Mapreduce | 0.014 | 0.008 | 0.005 | 0.004 | excellent parallelism |
| Threads | 0.011 | 0.011 | 0.011 | 0.011 | no effect of parallelism |
| Pmap (Default) | 3.968 | 52.120 | 45.869 | 46.113 | from bad to worse |
| Pmap (Batch_Size) | 3.932 | 4.269 | 4.544 | 4.382 | from bad to bad |

and with a longer function

## const M = 100
## @everywhere function longerfun(x::Int64)::Float64 xs=0.0; for o=1:(500^2); xs+= log(1.0+x); end; sqrt(xs); end#function

| Method | 1 nproc | 2 nproc | 4 nproc | 8 nproc | comments |
| Non Parallel | 197 | 197 | 197 | 1971 | as expected, constant |
| Shared Array | 197 | 102 | 55 | 31 | good parallelism |
| Shared Array, Mapreduce | 197 | 102 | 55 | 30 | goodparallelism |
| Threads | 197 | 197 | 197 | 197 | no effect of parallelism |
| Pmap (Default) | 198 | 112 | 59 | 30 | good parallelism |
| Pmap (Batch_Size) | 198 | 104 | 55 | 29 | good parallelism |

which I will also keep at http://julia.cookbook.tips/doku.php?id=parallel .

Does anyone know why threads has no parallel features?

24

Did you set the environment variable?

25

thanks. obviously not:

julia> Threads.nthreads()
1

I had incorrectly assumed that -p 8 would increase the number of threads, too, but it did not. duh!
instead, I had to do bash> export JULIA_NUM_THREADS=4.

is there an analog of julia> addprocs(3) ? I tried Threads.addthreads(3), but this did not work.

26

Threads have to be setup before the application is opened (at least for Julia), so I don’t think that’s possible.

27

I’ve found it pretty tricky to find a pseudocode example for sampling Monte Carlo simulations from a simulation function. Despite this topic being 3 years old, it’s still high on Google search results, hence the explanation here (abridged from @cecileane: notes).

Newbie tips

  • @parallel was deprecated since Julia v0.7 and replaced with @distributed in Distributed package.
  • @distributed OK for approx equal workload split per workers. Else consider pmap for load balancing and keeping idle workers busy!

Purpose: Minimal working example for independent nsims (embarassingly parallel e.g. Monte Carlo simulations) of roughly equal work or when nsims<available workers.

using Distributed
addprocs(2) # say

@everywhere begin
	using Distributed
	using SharedArrays
end

# define write vars beforehand
rwvar1 = SharedArray{Float64}(nsims)
[...]

# define read vars as normal
rvar1 = Array{Float64}(nsims)
[...]

# define simulationFunction as normal but made available to all workers
@everywhere function simulationFunction(rwvar1,...,rvar1,...)
	[...]
	return rwvar1[i] ...
end

# run it in parallel
@sync @distributed for i in 1:nsims
  out = simulationFunction(rwvar1[i],...,rvar1[i],...)
  rwvar1[i], ... = out
end

# save rwvar1,... as normal

To access the data in a SharedArray type, rwvar1.s is needed which then just returns a normal Array type which I found was easier to then save as a .jld for loading in R later.

1 Like
28

Any idea what algorithm JuliaLibm.log1p uses? log1p is pretty important, so if Base is under-performing, I might want to take a look.

29

AFAIK this was fixed by

https://github.com/JuliaLang/julia/pull/24750

30

Ah. I looked at that PR, but I did a git blame and saw that the code was from 2017, so I thought it couldn’t be the cause, and never realized that it was only merged in 2018.