Random number flow independent from the number of threads

General Usage
, ,
1

I am still trying to arrive to a workflow where I can set some multithreaded experiment random or fixed given some input depending of the needs.

The code above works for a given (fixed) number of threads, but if I let change the number of threads only the MersenneTwister implementation remains constant, while the StableRNG implementation returns an output that depends on the number of threads used. Do you know any workaround I could use ? I would like to keep using StableRNGs to keep my unit tests and output results independent on the specific Julia version.

using Random, StableRNGs, Statistics

function generateParallelRngs(rng::AbstractRNG, n::Integer)
    seeds = [rand(rng,100:18446744073709551615) for i in 1:n] # some RNGs have issues with too small seeds
    rngs  = [deepcopy(rng) for i in 1:n]
    return Random.seed!.(rngs,seeds)
end

function myFunction(rng = Random.GLOBAL_RNG)
    rngs       = generateParallelRngs(rng,Threads.nthreads()) # make new copy instances
    results    = Array{Float64,1}(undef,30)
    masterSeed = rand(rng,100:9999999999999)
    Threads.@threads for i in 1:30
        tsrng  = rngs[Threads.threadid()]    # Thread safe random number generator: one RNG per thread
        Random.seed!(tsrng,masterSeed+i*10)  # Here we let the seed depends on the i of the loop, not the thread: we get same results indipendently of the number of threads
        results[i] = sum(rand(tsrng, 1:1000,100))
    end
    overallResult = mean(results)
    return overallResult
end

myFunction(MersenneTwister(123)) # same result whatever the number of threads
myFunction(MersenneTwister(123)) # always same as previous line, as expected
myFunction(StableRNG(123))       # output depends on number of threads (i.e. `julia -t 2` ≠ `julia -t 4` outputs)
myFunction(StableRNG(123))       # always same as previous line, as expected
2

Just discovered that doing the master seeding before the generateParallelRngs solved the issue, but I still wonder why, and why the issue is only with the StableRNG…
EDIT: I’m an idiot. Of course rand in generateParallelRngs is called proportionally to the number of threads, so if I call rand again later, its output will depend on the number of threads. Now the question is the opposite: why the MersenneTwister implementation didn’t change…

3

It’s because generating a range of Int64 or of Int128 use different “caches” (arrays of ready-to-be-fetched random values) within a MersenneTwister. I.e, whether you use 2 or 4 values from say cache1, it doesn’t affect the state (next available value) of cache2.

2 Likes
4

Moreover, if you are re-seeding in each iteration, then there’s no point in seeding in generateParallelRngs.

1 Like