# Handling an optional rng parameter

**URL:** <https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895>\
**Category:** New to Julia\
**Created:** [February 17, 2019, 8:40am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895 "2019-02-17T08:40:10Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brian\_Thorne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian_thorne/32/6990_2.png) [@Brian\_Thorne](https://discourse.julialang.org/u/Brian_Thorne)\
**Post date:** [February 17, 2019, 8:40am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/1 "2019-02-17T08:40:10Z")

</div>

I’m developing my first Julia package [Paillier.jl](https://github.com/hardbyte/Paillier.jl) and wanted to know if I’m on the right track for handling an optional `rng`? All through the library should I be creating fallback function signatures that create a default `rng` if not given - like this:

```julia
generate_paillier_keypair(n_length=2048) = generate_paillier_keypair(RandomDevice(), n_length)
function generate_paillier_keypair(rng::AbstractRNG, n_length=2048)

```

Many thanks

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 17, 2019, 9:18am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/2 "2019-02-17T09:18:54Z")

</div>

I think that sounds reasonable, and is how it’s done within `Random` as well: [source here](https://github.com/JuliaLang/julia/blob/master/stdlib/Random/src/Random.jl#L245).

The duplication is not that pretty IMO. There are a few ways to get around it. One is to use keyword arguments to make both arguments optional so that you only need a single method. Another is to use macros/generated code (at the cost of less readable code). Finally, do what you’re doing now, but extract repeated constants/logic:

```julia
const DEFAULT_KEY_LENGTH = 2048

default_rng() = RandomDevice()

generate_paillier_keypair(n_length = DEFAULT_KEY_LENGTH) =
    generate_paillier_keypair(default_rng(), n_length)

function generate_paillier_keypair(rng::AbstractRNG, n_length = DEFAULT_KEY_LENGTH)

```

Btw, is it intended to create a new instance of `RandomDevice` for each call?

---

<div class="post-metadata">

**Author:** ![Brian\_Thorne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian_thorne/32/6990_2.png) [@Brian\_Thorne](https://discourse.julialang.org/u/Brian_Thorne)\
**Post date:** [February 17, 2019, 9:41am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/3 "2019-02-17T09:41:25Z")

</div>

Thanks for the advice. I think I’ll copy your example with the extracted logic.

Good question about creating new instances of `RandomDevice()` - I’m thinking no. On linux it looks to be a wrapper around `/dev/urandom` (which is what I want), and I would assume that just having one open file would be better.

---

<div class="post-metadata">

**Author:** ![Brian\_Thorne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian_thorne/32/6990_2.png) [@Brian\_Thorne](https://discourse.julialang.org/u/Brian_Thorne)\
**Post date:** [February 17, 2019, 10:04am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/4 "2019-02-17T10:04:24Z")

</div>

I ran into a weird behavior when trying to use a single `RandomDevice`. This code freezes/locks up while trying to run my unittests:

If I have the code:

```julia
default_rng = RandomDevice()

generate_paillier_keypair(n_length=DEFAULT_KEY_LENGTH) = generate_paillier_keypair(default_rng, n_length)
function generate_paillier_keypair(rng::AbstractRNG, n_length=DEFAULT_KEY_LENGTH)

```

So for now I’ll go back to create a new `RandomDevice` instance each call, but it would be good to understand what is going on.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [February 17, 2019, 12:27pm UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/5 "2019-02-17T12:27:00Z")

</div>

Unless you want to dispatch on the type of various RNGs, I would just use keyword arguments.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 17, 2019, 12:27pm UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/6 "2019-02-17T12:27:18Z")

</div>

Hmm. Strange. Yes, it’d be interesting to know what causes that. Perhaps you could try to create a minimal reproducible example, which in the best case will help you find what causes it, or at least you can post it here for us to test.

Are you using multiple threads by any chance? That will cause problems if using the same RNG:

```julia
julia> rng = RandomDevice()
RandomDevice(IOStream(<file /dev/urandom>), true)

julia> Threads.@threads for n = 1:100000 rand(rng) end

signal (11): Segmentation fault: 11
in expression starting at no file:0
jl_ios_get_nbyte_int at /Users/osx/buildbot/slave/package_osx64/build/src/sys.c:331
read at ./iostream.jl:402 [inlined]
rand at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.2/Random/src/RNGs.jl:29 [inlined]
...

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [February 17, 2019, 3:18pm UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/7 "2019-02-17T15:18:16Z")

</div>

Does constructing a new random device open `/dev/urandom` as a file? And are you perhaps doing this a very large number of times? If so, then perhaps your operating system is running out of file descriptors.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 17, 2019, 4:56pm UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/8 "2019-02-17T16:56:41Z")

</div>

It does:

```julia
julia> @btime RandomDevice()
ERROR: SystemError: opening file "/dev/urandom": Too many open files

```

But if I understand OP correctly, he’s saying that the problem occurs when he’s _not_ constructing a new `RandomDevice` for each call (i.e. when reusing the same RNG).

---

<div class="post-metadata">

**Author:** ![Brian\_Thorne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian_thorne/32/6990_2.png) [@Brian\_Thorne](https://discourse.julialang.org/u/Brian_Thorne)\
**Post date:** [February 24, 2019, 9:08am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/9 "2019-02-24T09:08:44Z")

</div>

Yeah that is the issue I ran into. I’ve tried to make a minimal demonstration but couldn’t recreate the issue until I started playing with the `include` statement.

My tests were running examples as well as unit tests - to ensure I didn’t break them:

```julia
# File: tests/runtests.jl
include("../src/Paillier.jl")
using Main.Paillier

...
@testset "Test raw_cryptosystem.jl example" begin
    @capture_stdout include("../examples/raw_cryptosystem.jl")
    @test 70 == decrypt(priv, c)
end

```

And the examples themselves were importing the `Paillier` module (not `Main.Paillier`) like the tests.

```julia
# file examples/encoded_numbers.jl
import Paillier
keysize = 2048
publickey, privatekey = Paillier.generate_paillier_keypair(keysize)
...

```

I was also running the tests from InteliJ using the `-L src/Paillier.jl` argument. When I change the `runtests.jl` file to `using Paillier` and don’t use the `-L` I get the same locking behavior (with no output).

So I guess I jumped to the `/dev/urandom` file but it is probably that I’m making a beginner mistake with my imports etc. Anyone able to point me in the right direction? How do people usually write and test examples as well as tests for a Julia library?

Many thanks for the help so far!

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [February 25, 2019, 3:39am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/10 "2019-02-25T03:39:20Z")

</div>

It should never be necessary for your test or example files to `include("../src/Paillier.jl")`, and you should not need to do `-L src/Paillier.jl` either.

Instead, since you have already created a nice `Project.toml` file, you should be able to load your package (via `import Paillier` or `using Paillier`) whenever you have that Project active. The way you activate a project in Julia is:

```julia
cd /whatever/Paillier
julia --project

```

or, if you are already in Julia:

```julia
pkg> activate /whatever/Paillier

```

(where you get the `pkg>` prompt by pressing `]` ).

I would recommend trying all of this in a terminal for now. I’m not sure what exactly intelliJ is doing, and it will be easier to save debugging that part of the puzzle for later. In a terminal, you should be able to do:

```julia
cd /whatever/Paillier
julia --project
julia> using Paillier

```

If that doesn’t work, I would remove all your includes until you just have a plain `Paillier/src/Paillier.jl` file with just `module Paillier` in it and then add your includes one by one.

Assuming that works, you should be able to change your `runtests.jl` to just do:

```julia
using Paillier # no include required

```

then you can try it with:

```julia
cd /whatever/Paillier
julia --project
pkg> test Paillier

```

(note that you don’t actually need to do the `cd` and `julia --project` steps every time. You can just keep the same Julia terminal open. I’m including them every time for completeness).

Assuming that works, you should be able to change your examples to also do:

```julia
using Paillier 

```

(or `import Paillier` if you prefer. There’s no difference in the way `using` and `import` look for packages).

And you should be able to run your examples with:

```julia
cd /whatever/Paillier
julia --project example.jl

```

Once that’s all working, we can figure out how IntelliJ factors into it.

---

<div class="post-metadata">

**Author:** ![Brian\_Thorne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian_thorne/32/6990_2.png) [@Brian\_Thorne](https://discourse.julialang.org/u/Brian_Thorne)\
**Post date:** [February 26, 2019, 7:26am UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/11 "2019-02-26T07:26:03Z")

</div>

Thanks so much for your help understanding this! Good idea to remove IntelliJ from the equation for now.

I’ve taken your advice with removing the `-L` and includes, and now I’m back to suspecting an issue with the `RandomDevice` is involved. If I run an example it still locks up but on Ctrl-C outputs the following:

```julia
[brian@hardbyte-nzxt Paillier.jl]$ julia --project examples/raw_cryptosystem.jl
^C
signal (2): Interrupt
in expression starting at /home/brian/dev/Paillier.jl/examples/raw_cryptosystem.jl:3
__select at /usr/bin/../lib/libc.so.6 (unknown line)
unknown function (ip: 0x7f60641025eb)
unknown function (ip: 0x7f6064104c9d)
ios_readprep at /usr/bin/../lib/libjulia.so.1 (unknown line)
jl_ios_get_nbyte_int at /usr/bin/../lib/libjulia.so.1 (unknown line)
read at ./iostream.jl:402 [inlined]
rand at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/RNGs.jl:29 [inlined]
rand! at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:235 [inlined]
rand! at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:231 [inlined]
rand! at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:231 [inlined]
macro expansion at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/generation.jl:325 [inlined]
macro expansion at ./gcutils.jl:87 [inlined]
rand at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/generation.jl:322
rand at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:219 [inlined]
n_bit_random_number at /home/brian/dev/Paillier.jl/src/utilities.jl:9
nbit_prime_of_size at /home/brian/dev/Paillier.jl/src/utilities.jl:17 [inlined]
generate_paillier_keypair at /home/brian/dev/Paillier.jl/src/keygeneration.jl:20
unknown function (ip: 0x7f603e89edd7)
jl_fptr_trampoline at /usr/bin/../lib/libjulia.so.1 (unknown line)
jl_apply_generic at /usr/bin/../lib/libjulia.so.1 (unknown line)
generate_paillier_keypair at /home/brian/dev/Paillier.jl/src/keygeneration.jl:10
unknown function (ip: 0x7f603e89e773)
jl_fptr_trampoline at /usr/bin/../lib/libjulia.so.1 (unknown line)
jl_apply_generic at /usr/bin/../lib/libjulia.so.1 (unknown line)
unknown function (ip: 0x7f606411c797)
unknown function (ip: 0x7f606411c540)
unknown function (ip: 0x7f606411d508)
unknown function (ip: 0x7f606411dbe4)
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7f6054a0a3ff)
unknown function (ip: (nil))
unknown function (ip: 0x7f606411e0ac)
unknown function (ip: 0x7f6063ffa3ac)
unknown function (ip: 0x7f6063fd255e)
jl_load at /usr/bin/../lib/libjulia.so.1 (unknown line)
unknown function (ip: 0x7f6058d4da32)
unknown function (ip: 0x7f6058d58b5b)
jl_apply_generic at /usr/bin/../lib/libjulia.so.1 (unknown line)
unknown function (ip: 0x7f6058d5815a)
unknown function (ip: 0x7f6058d58867)
jl_apply_generic at /usr/bin/../lib/libjulia.so.1 (unknown line)
unknown function (ip: 0x55b57596c6e3)
unknown function (ip: 0x55b57596c0a8)
__libc_start_main at /usr/bin/../lib/libc.so.6 (unknown line)
unknown function (ip: 0x55b57596c15d)
unknown function (ip: 0xffffffffffffffff)
Allocations: 875529 (Pool: 875327; Big: 202); GC: 1

```

When I change the `RandomDevice` line I get a working example:

```julia
_default_rng = RandomDevice()
# broken line (used for output above)
#default_rng() = _default_rng
# working line (used for output below)
default_rng() = RandomDevice() 

```

```julia
[brian@hardbyte-nzxt Paillier.jl]$ julia --project examples/raw_cryptosystem.jl
PublicKey(bits=1024, hash=7225169492579635362)
PrivateKey(hash=3823826835132048832)
decrypt(a) = 10
decrypt(b) = 50
decrypt(a + 5) = 15
decrypt(2a + b) = 70

```

A similar behavior from the package manager:

```julia
(Paillier) pkg> test Paillier
   Testing Paillier
 Resolving package versions...
^C
ERROR: LoadError: InterruptException:
Stacktrace:
 [1] read at ./iostream.jl:402 [inlined]
 [2] rand at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/RNGs.jl:29 [inlined]
 [3] rand! at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:235 [inlined]
 [4] rand! at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:231 [inlined] (repeats 
2 times)
 [5] macro expansion at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/generation.jl:325 [inlined]
 [6] macro expansion at ./gcutils.jl:87 [inlined]
 [7] rand(::Random.RandomDevice, ::Random.SamplerBigInt) at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/generation.jl:322
 [8] rand at /build/julia/src/julia-1.0.3/usr/share/julia/stdlib/v1.0/Random/src/Random.jl:219 [inlined]
 [9] n_bit_random_number(::Random.RandomDevice, ::BigInt) at /home/brian/dev/Paillier.jl/src/utilities.jl:9
 [10] nbit_prime_of_size at /home/brian/dev/Paillier.jl/src/utilities.jl:17 [inlined]
 [11] generate_paillier_keypair(::Random.RandomDevice, ::Int64) at /home/brian/dev/Paillier.jl/src/keygeneration.jl:20
 [12] generate_paillier_keypair(::Int64) at /home/brian/dev/Paillier.jl/src/keygeneration.jl:10

```

All this code is open source if that helps in any way - [https://github.com/hardbyte/Paillier.jl/tree/refactor-rng](https://github.com/hardbyte/Paillier.jl/tree/refactor-rng)

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [February 26, 2019, 2:46pm UTC](https://discourse.julialang.org/t/handling-an-optional-rng-parameter/20895/12 "2019-02-26T14:46:31Z")

</div>

I would guess this is a precompilation issue. By creating a single RandomDevice as a global variable in your module, that value is saved during precompilation of the module, but it may not actually be safe to do so (since that device might contain pointers or file handles which are not safe to use in another Julia process). When you define your function to return a new random device each time it’s called, the problem goes away because a new valid random device is constructed each time you ask for one.

If this is the problem, and you still want to share a single random device, then it’s easy to fix: you just need to delay construction of the random device until your module is actually loaded. You do that with an ` __init__ ` function inside your module:

```julia
module Foo

const _device = Ref{RandomDevice}()
default_rng = _device[]

function __init__ ()
  _device[] = RandomDevice()
end

```