CUDA.jl random number generation

Specific Domains GPU
1

Hi all,

I am trying to understand Random Numbers generated in CUDA. In particular which RNG is used in each version/case. If I understood this correctly:

using CUDA

v  = randn(Float64, N)

will use CUDA.jl interface, and in particular will make use of Philox RNG.

On the other hand, the code:

using CUDA

v = CuArray{Float64}(undef, N);
randn!(v)

will make use of the cuRAND library, and by default will use XORWOW.

Is this correct?

What is the canonical way to seed these generators?

Has this changed in different CUDA versions? If yes, is there a document where I can check the RNG used in each version?

Is there a canonical way to use a (good) RNG across CUDA versions?

Many thanks!

2

That simply generates a CPU array using the Random.jl stdlib. Maybe you’re confusing with doing that within a kernel, which will use a Philox-based RNG.

From the host, there’s multiple RNG choices. The main two ones are:

  • cuRAND.LibraryRNG: the cuRAND one
  • GPUArrays.RNG{CuArray} (also exposed as CUDA.RNG): a fallback for elements cuRAND can’t handle

CUDA.jl will automatically route towards one of those depending on the kind of array you’re working with (when calling CUDA.randn), or the element type you’re requesting (when calling CUDA.rand). But for explicit use you can always instantiate those RNGs directly and use them as the first argument to the Random.jl APIs. Seeding then also becomes very explicit; when doing CUDA.seed! we will automatically seed all RNGs that may be used.

3

Of course you are right… Sorry for the noise. I meant that:

v = randn(CUDA.default_rng(), Float64, N)

Will use CUDA.jl and the Philox RNG, while randn! calls fills the array using cuRAND.

Thanks!