# Random seeds in parallel computing
**URL:** https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899
**Category:** General Usage
**Tags:** parallel
**Created:** [September 19, 2020, 8:06am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899 "2020-09-19T08:06:30Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@AAL](https://discourse.julialang.org/u/AAL)
#### Post date: [September 19, 2020, 8:06am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/1 "2020-09-19T08:06:30Z")
I am trying to run a simulation in parallel and I am having trouble understanding how random numbers are generated in this setting. It seems like I can generate pseudorandom numbers in several cores with the same random seed:
```julia
using Distributed
addprocs(1)
@everywhere using SharedArrays, Random
A = SharedArray{Float64}(10,10)
rng = MersenneTwister(1)
@sync @distributed for i in 1:10
A[:,i] = rand(rng,10)
end
```
The output is equivalent when using a single core:
```julia
B = Array{Float64}(undef,10,10)
rng = MersenneTwister(1)
for i in 1:10
B[:,i] = rand(rng,10)
end
A==B #true
```
Why does specifying a single random seed work and why do I get the same result? I thought if two parallel processes accessed and modified the state of the same random number generator, I would get an error or at least a different result since it is accessed in a different order. Am I doing something wrong and my code is not really running in parallel?
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [September 19, 2020, 8:54am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/2 "2020-09-19T08:54:46Z")
Yes, you don’t run in parallel.
Without cmd line option `-p` I can reproduce your finding with `addprocs(1)`.
```julia
-p, --procs {N|auto} Integer value N launches N additional local worker processes
"auto" launches as many workers as the number of local CPU threads (logical cores)
```
You can see by filling the array with the worker IDs:
```julia
using Distributed
addprocs(1)
@everywhere using SharedArrays, Random
A = SharedArray{Int}(10,10)
rng = MersenneTwister(1)
@sync @distributed for i in 1:10
A[:,i] .= myid()
end
A
```
Only a single worker ID:
```julia
julia> A
10×10 SharedArray{Int64,2}:
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2
```
Adding e.g. `-p 4` on the command line OR doing e.g. `addprocs(4)` really adds some workers:
```julia
julia> A
10×10 SharedArray{Int64,2}:
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
2 2 2 3 3 3 4 4 5 5
```
With this you see the random numbers are the same for each worker ID, the RNG is seeded equally for each worker.
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [September 19, 2020, 9:09am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/3 "2020-09-19T09:09:11Z")
`nworkers()` tells you how many workers you are using:
```julia
julia> using Distributed
julia> addprocs(1)
1-element Array{Int64,1}:
2
julia> nworkers()
1
```
So nothing parallel.
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [September 19, 2020, 9:26am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/4 "2020-09-19T09:26:07Z")
> [@AAL](#):
>
> Why does specifying a single random seed work and why do I get the same result? I thought if two parallel processes accessed and modified the state of the same random number generator, I would get an error or at least a different result since it is accessed in a different order.
This I can’t answer and it seems to be complex. See this old discussion:
>
>
> How should parallel RNG be addressed? Codes that do parallel RNG:
>
> This is an MP…I code that does not seem to be maintained:
> http://sprng.cs.fsu.edu/
>
> A derivative of Mersenne Twister:
> http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/DC/dc.html
>
> At the very least,when randomize() is called, it should include the processor's number along with time to select a different seed on every processor.
And this one not so old:
> [@Brew a Parallel RNG?](https://discourse.julialang.org/t/brew-a-parallel-rng/30286):
>
> RNGs cannot be parallelized without modification, as the effectiveness of the randomness representation ceases if the sequences are related. Previous discussions have been found from But can we use some alternative solutions? Consider some ancient algorithms that are probably not in use nowadays(most written from the book Numerical Analysis by T. Sauer) # rng #random number generators #Computers are not capable of generating true random numbers #but can generate sequences with statistical …
What you actually need now is different random numbers for each worker but reproducible… (still looking)
This looks good:
```julia
using Distributed
addprocs(4)
@everywhere using SharedArrays, Random
A = SharedArray{Float64}(10,10)
@everywhere Random.seed!(myid())
@sync @distributed for i in 1:10
A[:,i] = rand(10)
end
```
---
### Author:  [@AAL](https://discourse.julialang.org/u/AAL)
#### Post date: [September 20, 2020, 6:46am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/5 "2020-09-20T06:46:16Z")
Thanks a lot for the answer. I thought nprocs() would be the number of parallel workers while nworkers() would exclude the main process. You are right that when using more workers `A!=B` because each worker starts with the same seed, so I get several copies of the same random numbers.
Specifying the seed with the worker id may be dangerous because it is not reproducible with different number of workers. The following is safer but it is less efficient by creating a new seed in every loop:
```julia
A = SharedArray{Float64}(10,10)
@sync @distributed for i in 1:10
rng = MersenneTwister(i)
A[:,i] = rand(rng,10)
end
```
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [September 20, 2020, 8:57am UTC](https://discourse.julialang.org/t/random-seeds-in-parallel-computing/46899/6 "2020-09-20T08:57:13Z")
> [@AAL](#):
>
> Specifying the seed with the worker id may be dangerous because it is not reproducible with different number of workers.
Absolutely true. But different number of workers does imply that it doesn’t reproduce. Reproducable with same conditions and environment of course. But even with same number of workers it may not reproduce, because the IDs may differ.
This should do:
```julia
using Distributed
addprocs(4)
@everywhere using SharedArrays, Random
function remote_seed()
ids=SharedArray{Int}(nworkers())
#now I assume that all workers will be used once:
@sync @distributed for i in 1:nworkers()
ids[i]=myid()
end
println(ids)
for i in 1:length(sort(ids))
remotecall(Random.seed!,ids[i],i)
end
end
remote_seed()
A = SharedArray{Float64}(10,10)
@sync @distributed for i in 1:10
A[:,i] = rand(10)
end
```
At least the seeding should be the same for each run.
As the distribution of the single tasks in a @distributed loop on the workers is not well defined it still may not reproduce the random numbers, but this could be achieved with calling `remotecall` for `rand` as done with `Random.seed!`.
Here ends my interest in the topic of reproducablilty of random numbers in a distributed environment 🙂