How to ensure reproducibility when using CUDA.rand()?

Specific Domains GPU
1

It seems that it is not supported to pass a specified rng as argument to rand in CUDA.jl.
The codes below raise an error:

using CUDA
rng = CUDA.RNG(42)
a = CUDA.rand(rng, 1)

ERROR: MethodError: no method matching rand(::CUDA.RNG, ::Int64)
You may have intended to import Base.rand
The function `rand` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  rand(::Integer, ::Integer...)
   @ CUDA E:\Softwares\Julia\.julia\packages\CUDA\x8d2s\src\random.jl:340
  rand(::Union{Type{Float32}, Type{Float64}, Type{UInt32}}, ::Integer, Integer...)
   @ CUDA E:\Softwares\Julia\.julia\packages\CUDA\x8d2s\src\random.jl:306
  rand(::Type, ::Integer, Integer...)
   @ CUDA E:\Softwares\Julia\.julia\packages\CUDA\x8d2s\src\random.jl:330
2

See

Summary: it’s Random.seed!(), not CUDA.RNG (where did you find that?)

3

Thanks for the link.
So the rng cannot be specified as an argument of CUDA.rand?
I tried the codes below:

Random.seed!(42)
a = CUDA.rand(1) # 0.24492861
Random.seed!(42)
b = CUDA.rand(1) # 0.19734985

It is still not reproducible…

4

Uhm, I think it’s CUDA.seed!, instead of Random.seed!

julia> using CUDA

julia> CUDA.seed!(42)

julia> CUDA.rand(Float32, 1)
1-element CuArray{Float32, 1, CUDA.DeviceMemory}:
 0.070020944

julia> CUDA.seed!(42)

julia> CUDA.rand(Float32, 1)
1-element CuArray{Float32, 1, CUDA.DeviceMemory}:
 0.070020944

Alternatively:

julia> rng = CUDA.RNG(2)
CUDA.RNG(0x00000002, 0x00000000)

julia> rand(rng, Float32, 1)
1-element CuArray{Float32, 1, CUDA.DeviceMemory}:
 0.59588563

julia> rng = CUDA.RNG(2)
CUDA.RNG(0x00000002, 0x00000000)

julia> rand(rng, Float32, 1)
1-element CuArray{Float32, 1, CUDA.DeviceMemory}:
 0.59588563
1 Like