# Random and threads

**URL:** <https://discourse.julialang.org/t/random-and-threads/65769>\
**Category:** General Usage\
**Tags:** random\
**Created:** [August 4, 2021, 1:35am UTC](https://discourse.julialang.org/t/random-and-threads/65769 "2021-08-04T01:35:59Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![gianmariomanca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gianmariomanca/32/25597_2.png) [@gianmariomanca](https://discourse.julialang.org/u/gianmariomanca)\
**Post date:** [August 4, 2021, 1:35am UTC](https://discourse.julialang.org/t/random-and-threads/65769/1 "2021-08-04T01:35:59Z")

</div>

Given the following snippet (assuming more than 1 thread):

```julia
const NC = 3000;
const z_vec = zeros(NC);
Threads.@threads for i = 1:NC
    z_vec[i] = randn()
end

```

What statistical properties we should expect in 1.6.2 for z\_vec?  
Are the statistical properties expected to be as good as a single call z\_vec = randn(NC)?  
Would the answer be the same for 1.7?

Basically trying to understand if it’s safe to add Threads.@threads without too much thinking for a quick speedup or I need to get serious about selecting seeds etc.

---

<div class="post-metadata">

**Author:** ![gianmariomanca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gianmariomanca/32/25597_2.png) [@gianmariomanca](https://discourse.julialang.org/u/gianmariomanca)\
**Post date:** [August 4, 2021, 3:09pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/2 "2021-08-04T15:09:37Z")

</div>

Well, I can always allocate also preallocate the random sequence

```julia
const NC = 3000;
const z_vec = zeros(NC);
const rr = randn(NC);
Threads.@threads for i = 1:NC
    z_vec[i] = rr[i]
end

```

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 4, 2021, 5:04pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/3 "2021-08-04T17:04:05Z")

</div>

I would like to know the answer, though.  
It is not always practical to pre-draw all random variables. I sometimes simulate a number of agents’ histories. Each agent draws quite a few random variables.  
My approach is to pre-draw the seeds for the random number generators that are passed to each agent. I am curious whether this introduces any issues (e.g., draws being correlated across agents where they should be independent).

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 4, 2021, 6:32pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/4 "2021-08-04T18:32:39Z")

</div>

My usual take on this is that if the generation of the seeds is high quality most likely the correlations are irrelevant. For this kind of thing it can be useful to utilize OS cryptorng facilities. Like /dev/urandom

You can try running experiments, by seeding two sequences and comparing the outputs with stats tests over and over. For example use the distance correlation from EnergyStatistics.jl

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 4, 2021, 6:50pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/5 "2021-08-04T18:50:59Z")

</div>

> [@gianmariomanca](#):
>
> What statistical properties we should expect in 1.6.2 for z\_vec?

On 1.6, this should basically be the same as for the single threaded version since there’s only one global RNG.

> [@gianmariomanca](#):
>
> Are the statistical properties expected to be as good as a single call z\_vec = randn(NC)?

`randn` works by calling `rand` internally, it’s just faster because it can do batch processing and doesn’t have to do work by calling `rand` for each index (it’s a little trickier than that in reality, but that’s the gist of it). They share the underlying normal distribution. You can of course call `randn()` for each thread individually, but if you need the full random matrix anyway, why not call `randn(NC)` in the first place and skip the threaded filling?

> [@gianmariomanca](#):
>
> Would the answer be the same for 1.7?

The upcoming 1.7 has an [independent RNG per task](https://github.com/JuliaLang/julia/pull/40546), but that doesn’t change anything about the distribution characteristics, which is still normally distributed.

---

<div class="post-metadata">

**Author:** ![gianmariomanca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gianmariomanca/32/25597_2.png) [@gianmariomanca](https://discourse.julialang.org/u/gianmariomanca)\
**Post date:** [August 4, 2021, 7:21pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/6 "2021-08-04T19:21:12Z")

</div>

@Sukera , I’m not sure if you are reading the question too literally. As @hendri54 mentioned is not always practical to pre-draw all random sequences you need, it can easily get out of hand.

The whole point is “user-friendliness” and reducing cognitive load: one less thing I have to worry about to focus on what I really care about.

> [@dlakelan](#):
>
> My usual take on this is that if the generation of the seeds is high quality most likely the correlations are irrelevant. For this kind of thing it can be useful to utilize OS cryptorng facilities. Like /dev/urandom

you can rephrase the original question as “would Julia do that (or something similar) automatically, with some guarantees, or not?”

If 1.7 has independent RNGs per task, the question seems particularly relevant for 1.7 then. If all task were to start with the same seed, it would be a big issue. I’m assuming they will not, but it would be nice to know something more about the mechanism and the “guarantees”.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 4, 2021, 7:36pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/7 "2021-08-04T19:36:33Z")

</div>

> [@gianmariomanca](#):
>
> If 1.7 has independent RNGs per task, the question seems particularly relevant for 1.7 then. If all task where to start with the same seed, it would be a big issue. I’m assuming they will not, but it would be nice to know something more about the mechanism and the “guarantees”.

I think you should always run `Random.seed!()` it sounds like in 1.7 this will set the seed of your task-local RNG. The question is how you should do this? I don’t think you should rely on Julia to do it for you. You can however use `Random.RandomDevice` to get OS level crypto entropy to help you.

```julia
sd = rand(Random.RandomDevice(),UInt)
Random.seed!(sd)

```

should do it

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 4, 2021, 7:47pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/8 "2021-08-04T19:47:24Z")

</div>

That’s not necessary. The task local RNG is seeded from the parent tasks’ RNG automatically on creation of the task.

You can check [`jl_new_task`](https://github.com/JuliaLang/julia/blob/e76c9dad42c137c7812fb1970ecc23166f78ee1f/src/task.c#L705), [`rng_split`](https://github.com/JuliaLang/julia/blob/e76c9dad42c137c7812fb1970ecc23166f78ee1f/src/task.c#L681) and [`jl_tasklocal_genrandom`](https://github.com/JuliaLang/julia/blob/e76c9dad42c137c7812fb1970ecc23166f78ee1f/src/task.c#L648) in [`src/task.c`](https://github.com/JuliaLang/julia/blob/v1.7.0-beta3/src/task.c) for how this is done, but from what I can tell this is secure and preserves all properties of the regular RNG since it’s the same kind of RNG anyway. It also works the exact same way [`seed!`](https://github.com/JuliaLang/julia/blob/5118a1babf9d2963f31e98e427cc99a969d5ca46/stdlib/Random/src/Xoshiro.jl#L118) does.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 4, 2021, 7:58pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/9 "2021-08-04T19:58:45Z")

</div>

Interesting. Seems likely you can ignore this issue for general purposes, but if I cared about independence in a strong way, I’d still be seeding from crypto source. At the very least you can disclaim “there’s nothing about my seeding method that is under my control”. For stuff that I sometimes do, this is potentially important (ie. Forensics)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 4, 2021, 8:06pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/10 "2021-08-04T20:06:16Z")

</div>

> [@dlakelan](#):
>
> I’d still be seeding from crypto source

`Random.seed!` (I’ve added the link above) already does this, it gets a `RandomDevice()` as the parent RNG since that’s an RNG we can “trust” on some level, so you don’t have to get it yourself and draw from it manually. (I guess that’s what you were asking about with drawing from `/dev/urandom`, @gianmariomanca?)

Seeding from a crypto source is not really necessary when writing code that does not require it. In those cases, it’s more important to ensure different properties (such as uniformity or performance) from the regular RNG, which Xoshiro in the upcoming 1.7 should definitely satisfy.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 4, 2021, 8:10pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/11 "2021-08-04T20:10:03Z")

</div>

Perfect. that’s great. So if you want crypto random seed it looks like you’d just do

```julia
Random.seed!()

```

Thanks!

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 4, 2021, 8:11pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/12 "2021-08-04T20:11:30Z")

</div>

I haven’t looked into how the first task is spawned in 1.7, but I’d guess that’s how the initial RNG of the first task that runs when you start `julia` is started is seeded already since the first call to `rand()` is always different 🙂

---

<div class="post-metadata">

**Author:** ![blackeneth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blackeneth/32/10353_2.png) [@blackeneth](https://discourse.julialang.org/u/blackeneth)\
**Post date:** [August 5, 2021, 5:22am UTC](https://discourse.julialang.org/t/random-and-threads/65769/13 "2021-08-05T05:22:43Z")

</div>

Watch out for the [birthday paradox](https://www.johndcook.com/blog/2016/01/29/random-number-generator-seed-mistakes/), however.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [August 5, 2021, 4:24pm UTC](https://discourse.julialang.org/t/random-and-threads/65769/14 "2021-08-05T16:24:51Z")

</div>

Yes, however it looks like seed! grabs 4 UInt64s from /dev/urandom, there’s 256 bits of random seed then, and the birthday paradox would start to kick in when you’re spawning on the order of 2^128 threads 😉 so basically no issues there.