# Best practices for parallel generation of pseudo-random numbers

**URL:** <https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504>\
**Category:** Modelling & Simulations\
**Tags:** question\
**Created:** [July 18, 2019, 3:14pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504 "2019-07-18T15:14:50Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [July 18, 2019, 3:14pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/1 "2019-07-18T15:14:50Z")

</div>

I’m relatively new to the world of simulation and am running some discrete event simulations using SimJulia.jl and Distributions.jl for a graduate-level class. I noticed that the Wikipedia article for Mersenne Twister (which Julia uses under the hood for PRNG mentions that there are issues with independence if using multiple instances differing only in seed value. There’s also [a paper on an approach to solving the problem](https://en.wikipedia.org/wiki/Mersenne_twister#cite_note-40), but I haven’t yet read it in depth.

Is there a canned Julia solution to generating random numbers in parallel so that multiple simulation replications are independent? If I’m running simulations in parallel using multithreading and just have all draw from `GLOBAL_RNG`, is that sufficient?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [July 18, 2019, 3:54pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/2 "2019-07-18T15:54:35Z")

</div>

Create one RNG per thread. In the future `rand()` will likely be thread safe.

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [July 18, 2019, 4:02pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/3 "2019-07-18T16:02:31Z")

</div>

Good to know that `rand` isn’t currently thread-safe. I’m also looking on advice for dealing with (from Wikipedia):

> Multiple instances that differ only in seed value (but not other parameters) are not generally appropriate for [Monte-Carlo simulations](https://en.wikipedia.org/wiki/Monte-Carlo_simulation) that require independent random number generators, though there exists a method for choosing multiple sets of parameter values. [[40]](https://en.wikipedia.org/wiki/Mersenne_twister#cite_note-40)[[41]](https://en.wikipedia.org/wiki/Mersenne_twister#cite_note-41)

So best practices for seeding said RNGs.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [July 18, 2019, 4:06pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/4 "2019-07-18T16:06:43Z")

</div>

use the `randjump` function from the `Future` module.  
However, if you use Mersenne Twister the risk of collision, due to birthday paradox, of simply using different (possibly randomized) seeds is very low due to its long period.

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [July 18, 2019, 4:26pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/5 "2019-07-18T16:26:08Z")

</div>

Thanks. What value(s) should be used for `steps` in randjump?

EDIT: found the section in the docs that mentions this I guess `big(10)^20` is standard since it’s precomputed?

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [July 18, 2019, 4:45pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/6 "2019-07-18T16:45:38Z")

</div>

Yes `big(10)^20` is enough in practice (on my laptop I would have to wait for over 10 000 years to get an overlap).

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [July 18, 2019, 4:47pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/7 "2019-07-18T16:47:58Z")

</div>

A secondary question as a point of curiousity: does anyone know why `randjump` lives in `Future` instead of in `Random`? As far as I can tell, there’s been no `randjump` function outside of `Future` available in any Julia 1.x series release.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [July 18, 2019, 4:58pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/8 "2019-07-18T16:58:51Z")

</div>

There was `randjump` with a bit different API (but essentially the same function) in `Random` in Julia 0.6 and earlier.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [July 18, 2019, 5:21pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/9 "2019-07-18T17:21:51Z")

</div>

Just to be sure, have you looked at the [docs](https://docs.julialang.org/en/v1/manual/parallel-computing/#The-@threads-Macro-1)?

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [July 18, 2019, 5:31pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/10 "2019-07-18T17:31:33Z")

</div>

I have now. Previously I just looked at the docs for the Random module and so had missed the section with `randjump`.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 25, 2019, 3:21pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/11 "2019-07-25T15:21:47Z")

</div>

Very odd… _ **just** _ stumbled on such packages by accident.

I am not very well versed in this, but I found 2 packages that might be similar to what you are looking for:

- [https://github.com/grayclhn/ParallelRNGs.jl](https://github.com/grayclhn/ParallelRNGs.jl) : Random Number Generators for Julia, designed for a parallel computing environment
- [https://github.com/awllee/RNGPool.jl](https://github.com/awllee/RNGPool.jl) : A pool of random number generators for multi-threaded applications
- (It looks like there might be more)

_ **NOTE:** _ these are probably unpublished (you would have to manually add the packages using their `https://` paths).

You can use the “JuliaObserver” tool to find such packages:  
 → [https://juliaobserver.com/packages](https://juliaobserver.com/packages)

You might want to search for terms, such as “pseudo” or “RNG”.

---

<div class="post-metadata">

**Author:** ![isaacsas](https://avatars.discourse-cdn.com/v4/letter/i/f6c823/32.png) [@isaacsas](https://discourse.julialang.org/u/isaacsas)\
**Post date:** [July 25, 2019, 3:35pm UTC](https://discourse.julialang.org/t/best-practices-for-parallel-generation-of-pseudo-random-numbers/26504/12 "2019-07-25T15:35:08Z")

</div>

[RandomNumbers.jl](https://github.com/sunoru/RandomNumbers.jl) includes several generators designed to allow multiple simultaneous streams like PCG and Random123. They aren’t the fastest, but each can combine a seed with an “id” for each stream. I haven’t used the Julia interface/implementations of them yet, but I assume they expose similar functionality.