Disable default rng

General Usage
,
1

In a test suite I want to make sure that the default RNG is never called (clarification: even indirectly, somewhere within a package), ie that all calls to rand & friends have an explicit rng as their first argument.

This old topic has some hacks, but the one I suggested is outdated (GLOBAL_RNG is no longer part of the API, and code may be threaded), and the other relies on redefining rand, which does not help as the code I am testing is in a package I don’t want to modify like this (recall, this is for CI).

An API that would allow setting the default RNG to an instance of

struct DummyRNG <: AbstractRNG end # I define no methods so it errors

would solve the problem, but I there is no default_rng!.

I am interested in all suggestions which would work in 1.10+, even hacks.

1 Like
2

A hack would be to overwrite the method definition of Random.default_rng():

julia> Random.default_rng() = nothing

julia> rand()
ERROR: MethodError: no method matching rand(::Nothing, ::Type{Float64})
The function `rand` exists, but no method is defined for this combination of argument types.
[...]

julia> rand(Xoshiro())
0.7427065090263826
4 Likes
3

Is the intention to make sure that CI runs are deterministic by disallowing calls to rand involving that RNG?

1 Like
4

I thought of that too, but unfortunately @testset also uses it:

julia> Random.default_rng() = error("disabled for testing")  # overwrite so it errors

julia> @testset "foo" begin
       @test 1 + 1 == 2
       end
ERROR: disabled for testing
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] default_rng()
   @ Main ./REPL[36]:1
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/Test/src/Test.jl:1698 [inlined]
 [4] top-level scope
   @ REPL[37]:2

(if you return nothing instead of erroring, it will just complain that it cannot copy it, etc, the setup & teardown code is quite hard to fool)

1 Like
5

No, the intention is to test that all computations that use random numbers do so with an explicit RNG.

1 Like
6

I see - you could create a copy of the default rng before & after the test and see whether it changed. That won’t catch rng in spawned tasks (the splitting RNG used for that is not exposed in the API, it’s hidden state in the runtime), but should cover any calls to rand in the original task.

Disallowing external RNG entirely is unfortunately quite difficult to do without some form of hook into the runtime.

3 Likes
7

Btw, with “deterministic” I basically meant the same thing as you did - if all RNG is only controlled by the instance that’s being passed in explicitly, passing in the same exact RNG (including internal state!) should lead to the same exact result when the test is executed twice. If the results differ, the test must have accessed some other RNG/source of non-determinism.

Detecting this kind of non-determinism is an upcoming feature of the next release of Supposition.jl, which is currently WIP. It’s not a drop-in solution though.

2 Likes
8

Here is a version which at least doesn’t completely break @testset:

Random.rand(::TaskLocalRNG, ::Random.SamplerType{UInt64}) = error("disabled")

This is the primary generating method for TaskLocalRNG, so it will block any other rand method.

Compared to the excellent other suggestion overwriting default_rng(), this will also catch uses in library via GLOBAL_RNG, which is set at startup to TaskLocalRNG() (like in somemethod(rng=Random.GLOBAL_RNG)).

EDIT: for clarification, GLOBAL_RNG was never part of the API, but was obvioulsy used by libraries. The API is now default_rng(), but to not break these libraries, GLOBAL_RNG was set equal to TaskLocalRNG().

1 Like
9

You can hack around that by returning your DummyRNG() instance, and defining:

using Random
struct DummyRNG <: AbstractRNG end
Random.default_rng() = DummyRNG()
Base.copy(rng::DummyRNG) = rng
Base.copy!(::DummyRNG, rng::AbstractRNG) = nothing
1 Like
10

Can you please explain why this works? Is it because implicitly, default_rng() returns a TaskLocalRNG?

(It looks like a reasonable workaround, I am just checking if I am relying on something that is documented, or internals)

11

Yes, default_rng() returns TaskLocalRNG(), probably since TaskLocalRNG() was introduced, so is true in particular since v1.10 (which is what you wanted to support iirc). It’s not a contract though, and could change in the future, but I don’t see a change coming soon.

I think default_rng() was introduced when we started to have a thread-safe “global RNG” implemented with multiple MersenneTwister objects, once per thread, in v1.13. The old undocumented Random.GLOBAL_RNG was not the correct object to use anymore; in contrast, default_rng() could look-up the thread ID and return a valid object.

1 Like