Getting Reproducible Results in Distributed Computing with Randomness

General Usage
, ,
1

I’m stuck on how to set different random seeds for each worker in distributed computation.

using Distributed, SharedArrays

# Start Julia with multiple workers
addprocs()

@everywhere using Random

# Function to set a unique random seed for each worker
@everywhere function set_worker_seed()
    worker_id = myid() - 1  # Worker IDs start from 2, so we subtract 1
    seed = 42 + 1000*worker_id  # Adjust the base seed value as needed
    Random.seed!(seed)
    println("Worker $worker_id seed: $seed")
end

# Set a unique random seed for each worker
for p in workers()
    @spawnat p set_worker_seed()
end

v = SharedArray{Float64}(10)
# Define a function that uses the random number generator
@sync @distributed for i=1:10
    v[i] = rand()
end
println(v)

My hope is to set different random seeds for each worker (so that there is no overlap in the value of v for each element) and make the vector v reproducible every time I reset the random seed.
But it turns out that the value of v changes for each run in my current code. Do you have any solutions on how to fix this issue?

Thank you!

2

This seems to be due to Tasks: don't advance task RNG on task spawn by StefanKarpinski · Pull Request #49110 · JuliaLang/julia · GitHub

In the meantime, consider defining an RNG on each worker explicitly, and using that in any random function. They should all accept an RNG as their first argument.

3

Thank you!
Putting MersenneTwister in the rand() function seems to provide what I wanted to do.

using Distributed, SharedArrays

# Start Julia with multiple workers
addprocs()

@everywhere using Random

rng = Vector{MersenneTwister}(undef,nworkers())
for i=1:nworkers()
    rng[i] = MersenneTwister(42+i*1000)
end

v = SharedArray{Float64}(10)
@sync @distributed for i=1:10
    v[i] = rand(rng[myid()-1])
end
println(v)