Best practices for parallel generation of pseudo-random numbers

Specific Domains Modelling & Simulations
1

I’m relatively new to the world of simulation and am running some discrete event simulations using SimJulia.jl and Distributions.jl for a graduate-level class. I noticed that the Wikipedia article for Mersenne Twister (which Julia uses under the hood for PRNG mentions that there are issues with independence if using multiple instances differing only in seed value. There’s also a paper on an approach to solving the problem, but I haven’t yet read it in depth.

Is there a canned Julia solution to generating random numbers in parallel so that multiple simulation replications are independent? If I’m running simulations in parallel using multithreading and just have all draw from GLOBAL_RNG, is that sufficient?

2 Likes
2

Create one RNG per thread. In the future rand() will likely be thread safe.

4 Likes
3

Good to know that rand isn’t currently thread-safe. I’m also looking on advice for dealing with (from Wikipedia):

Multiple instances that differ only in seed value (but not other parameters) are not generally appropriate for Monte-Carlo simulations that require independent random number generators, though there exists a method for choosing multiple sets of parameter values. [40][41]

So best practices for seeding said RNGs.

4

use the randjump function from the Future module.
However, if you use Mersenne Twister the risk of collision, due to birthday paradox, of simply using different (possibly randomized) seeds is very low due to its long period.

3 Likes
5

Thanks. What value(s) should be used for steps in randjump?

EDIT: found the section in the docs that mentions this I guess big(10)^20 is standard since it’s precomputed?

1 Like
6

Yes big(10)^20 is enough in practice (on my laptop I would have to wait for over 10 000 years to get an overlap).

3 Likes
7

A secondary question as a point of curiousity: does anyone know why randjump lives in Future instead of in Random? As far as I can tell, there’s been no randjump function outside of Future available in any Julia 1.x series release.

8

There was randjump with a bit different API (but essentially the same function) in Random in Julia 0.6 and earlier.

9

Just to be sure, have you looked at the docs?

1 Like
10

I have now. Previously I just looked at the docs for the Random module and so had missed the section with randjump.

11

Very odd… just stumbled on such packages by accident.

I am not very well versed in this, but I found 2 packages that might be similar to what you are looking for:

NOTE: these are probably unpublished (you would have to manually add the packages using their https:// paths).

You can use the “JuliaObserver” tool to find such packages:
→ https://juliaobserver.com/packages

You might want to search for terms, such as “pseudo” or “RNG”.

12

RandomNumbers.jl includes several generators designed to allow multiple simultaneous streams like PCG and Random123. They aren’t the fastest, but each can combine a seed with an “id” for each stream. I haven’t used the Julia interface/implementations of them yet, but I assume they expose similar functionality.

1 Like