How to set the seed properly in CURAND

Specific Domains GPU
1

I just found it’s a little tricky for me to set the seed in CuArrays.CURAND

julia> using CuArrays, CuArrays.CURAND

julia> s = CuArrays.CURAND.create_generator()
CuArrays.CURAND.RNG(Ptr{Nothing} @0x00000000054faec0, 100)

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.74021935
 0.9209938 
 0.03902049

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.9689629 
 0.92514056
 0.4463501

Obviously, setting the same seed s doesn’t work.

After reading the source code, I found there’s a function named set_pseudo_random_generator_seed:

julia> CuArrays.CURAND.set_pseudo_random_generator_seed(s, 123)
0x00000000

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.048817795
 0.57627815 
 0.8071622  

julia> CuArrays.CURAND.set_pseudo_random_generator_seed(s, 123)
0x00000000

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.036559016
 0.5210836  
 0.61308384

Still not working.

After googling I found this answer:

And yes, it seems works!

julia> CuArrays.CURAND.set_generator_offset(s, 0)
0x00000000

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.48431787
 0.40131402
 0.79087174

julia> CuArrays.CURAND.set_generator_offset(s, 0)
0x00000000

julia> rand(s, Float32, 3)
3-element CuArray{Float32,1}:
 0.48431787
 0.40131402
 0.79087174

I’m not very familiar with CURAND. Can anyone help to point out why I need to call set_generator_offset here (is it the right way to do so)? And what’s the meaning of set_pseudo_random_generator_seed?

Found another resource. Hope it is useful for someone else.

2

The curand documentation explains the offset briefly:

https://docs.nvidia.com/cuda/curand/host-api-overview.html#generator-options

To get the same random numbers you’d need the same seed and the same offset, as you’ve found. The seed defines the sequence, and the offset allows you to skip to a point in the sequence.

It does seem a bit surprising that setting the seed doesn’t also reset the offset to zero.

1 Like
3

Hi,

is this still working for you? In my case, the very first line

using CuArrays, CuArrays.CURAND
s = CuArrays.CURAND.create_generator()

spits

UndefVarError: create_generator not defined

Stacktrace:
 [1] getproperty(::Module, ::Symbol) at ./Base.jl:13
 [2] top-level scope at In[14]:2

so I can not initialize my random numboos :frowning:
But I’m 100% sure I’m missing something, so any help will be much appreciated.
BTW my CuArrays etc things are working well on my Julia 1.2.0 boxes… pretty well I must say! More than 100 times faster that usual Julia, which I start to seriously believe is not that fast as people claims…

Thanks in advance,

Ferran.

4

@findmyway made a PR that puts these calls in seed!, so there’s no need to call internal methods anymore: https://github.com/JuliaGPU/CuArrays.jl/pull/344.

1 Like
5

Oh, thanks, that now works as expected.
May I tell you that you guys are doing a great work with this CUDA ports to Julia? It’s been the first time that I can do real and useful CUDA programming without feeling it it a big mess :slight_smile:

2 Likes