# Why does increasing the number of identical parallel operations increases each operation time?

**URL:** <https://discourse.julialang.org/t/why-does-increasing-the-number-of-identical-parallel-operations-increases-each-operation-time/57014>\
**Category:** Performance\
**Tags:** linearalgebra, distributed\
**Created:** [March 12, 2021, 11:08am UTC](https://discourse.julialang.org/t/why-does-increasing-the-number-of-identical-parallel-operations-increases-each-operation-time/57014 "2021-03-12T11:08:36Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 12, 2021, 1:30pm UTC](https://discourse.julialang.org/t/why-does-increasing-the-number-of-identical-parallel-operations-increases-each-operation-time/57014/4 "2021-03-12T13:30:21Z")

</div>

> [@stevengj](#):
>
> I’m not sure what you’re referring to?

I am not sure either, but I have experienced problems with multi-threading and random number generators before, and I have seen this arising repeatedly here. I think the problem can be associated with the fact that if one wants to generate a single sequence of random numbers, all threads will be accessing the same seed sequentially, thus there is a problem there if the generation of the random numbers is a limiting step. Anyway, I am not sure when this actually applies, and when it does not.

Here is one post which I could find referring to this kind of problem:

> [@Number of threads available seems random (using @spawn)](https://discourse.julialang.org/t/number-of-threads-available-seems-random-using-spawn/44594/5):
>
> I think the poor scaling that you’re seeing is because the default global rng can suffer from poor performance when called from multiple threads. IIRC, the reason was due to cache invalidation (the default global rng for each thread is stored next to the others in an array without any padding, so when one thread changes the rng state, it forces the cache on all other cores to reload… or something like that.) [me@redmi ~]$ time ~/julia/bin/julia -t2 /tmp/error.jl 2 [DateTime("2020-08-09T15:24:2…

---

_[View the full topic](https://discourse.julialang.org/t/why-does-increasing-the-number-of-identical-parallel-operations-increases-each-operation-time/57014)._