# Use of Atomic variables in julia

**URL:** <https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511>\
**Category:** New to Julia\
**Tags:** question\
**Created:** [March 7, 2017, 2:15am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511 "2017-03-07T02:15:52Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ward9250](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ward9250/32/42768_2.png) [@Ward9250](https://discourse.julialang.org/u/Ward9250)\
**Post date:** [March 7, 2017, 2:15am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/1 "2017-03-07T02:15:52Z")

</div>

I’ve been a julia user for a long time, but I’m only just looking at multithreaded programming in general and wanted to play with julia to understand it a bit more. One of the things I wanted to use to learn a bit was some kind of multithreaded counting function:

```julia
function mycount(n)
    c = 0
    for i in 1:n
        c += i
    end
    return c
end

```

From what I understand, `c`, being a variable incremented frequently by all the threads, would mean if you were using locks and synchronisation to work out who (and by who I colloquially mean which thread) can write to it at one time, would make threads wait such that you might as well be doing things single threaded. But I’ve read the alternative is to use “atomic” variables, which I see are even a julia type in Threads. So really what I’d like to know is, in a nutshell how to atomic variables help solve this problem, is it better than using locks, and more julia specific - can one make sure the toy counter above is safe just by replacing `c` with an `Atomic` type and proceeding as normal, or must I also use this function I’ve found in the Threads docs: `Threads.atomic_add!`?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 7, 2017, 3:27am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/2 "2017-03-07T03:27:39Z")

</div>

No you shouldn’t use atomic variables. They are not meant for this and will essentially serialize everything. The correct way is to use a thread local counter and combine them together later.

It is not trivial to do all that in julia at those point and since you have not had previous experience with multithreaded programming, I don’t recommand you to start threaded programming in julia at this point.

---

<div class="post-metadata">

**Author:** ![Ward9250](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ward9250/32/42768_2.png) [@Ward9250](https://discourse.julialang.org/u/Ward9250)\
**Post date:** [March 8, 2017, 1:50am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/3 "2017-03-08T01:50:05Z")

</div>

I was talking with a colleague today about threads, and he said it’s probably better to make an array of length = nthreads, and each thread then has a local count in that array that is the reduced by a sum at the end, and I can see something similar in the docs where each thread writes to a different position in an array. This sounds a lot like your thread local counter? I was also told if computation of the count increment was quite time-consuming, then the use of atomic variables wouldn’t slow things down to much so it would be fine, but if it the operation is quick anyway (which it is), indeed then using the atomics would mean it might as well be serial.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 8, 2017, 2:08am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/4 "2017-03-08T02:08:49Z")

</div>

> [@Ward9250](#):
>
> he said it’s probably better to make an array of length = nthreads, and each thread then has a local count in that array that is the reduced by a sum at the end

That’ll actually likely be almost equally bad due to false sharing (google will give you a lot of good results).

> [@Ward9250](#):
>
> I was also told if computation of the count increment was quite time-consuming,

Well, define “time-consuming”. Atomic read-modify-write (RMW) operations are in general **really** expensive. Just to give you an idea, on my laptop, `sin(1.0)` takes `6ns` and an atomic increment of `1` takes `24ns`. One or two libm functions is well above my usual definition of “time-consuming” loop body since they are hundreds time more expensive than simple arithmetics but that’s not remotely enough to make the atomic increment overhead “fine”.

Also note that the `24ns` is single threaded measurement, if you are doing atomic increment from multiple threads, it’ll be even slower by few times due to cacheline ping-pong, same reason why you shouldn’t use an dense array as thread local counter.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 8, 2017, 2:19am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/5 "2017-03-08T02:19:26Z")

</div>

And as another example, SIMD loop for adding an array of 400 `Float64` or ~640 `Int64` also takes `24ns` on the same computer. If this can give you a better idea of how much computation you need in the loop to make atomic increment relatively cheap.

---

<div class="post-metadata">

**Author:** ![Ward9250](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ward9250/32/42768_2.png) [@Ward9250](https://discourse.julialang.org/u/Ward9250)\
**Post date:** [March 8, 2017, 11:12am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/6 "2017-03-08T11:12:56Z")

</div>

I see so if my googling is correct, the array would be in one area of the cache, and multiple threads modifying it still create issues. If this is the case why do the docs have the zeros example in it?

```julia
a = zeros(10)
Threads.@threads for i = 1:10
    a[i] = Threads.threadid()
end

```

I know in C11, you can use ‘thread\_local’ from threads.h for local variables. You alluded to that thread local variables can be done in julia, but it is not trivial, where can I read how it is currently done? Rest assured I’m not asking for a practical reason - I don’t have any desire to seriously use threads in production julia code whilst it’s still experimental, I’m just trying to get a better appreciation of julia.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 8, 2017, 1:55pm UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/7 "2017-03-08T13:55:07Z")

</div>

> [@Ward9250](#):
>
> a[i] = Threads.threadid()

The loop will be partitioned to minimize that.

> [@Ward9250](#):
>
> how it is currently done

You can mimic it with an array. Just need to be careful about padding or use indirections.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 27, 2017, 11:50pm UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/8 "2017-10-27T23:50:09Z")

</div>

```julia
function nothread_test(v)
  for i = 1:length(v)
    @inbounds v[i] = rand()
  end
  sum(v)
end

function thread_test(v)
  Threads.@threads for i = 1:length(v)
      @inbounds v[i] = rand()
  end
  sum(v)
end

@benchmark nothread_test!(rand(250_000_000))
@benchmark thread_test!(rand(250_000_000))

```

I just tried to use threads (n = 4) to populate an array and sum it. I expected the Threads version to run faster. But I don’t think I figured out the way to do it just yet.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [October 28, 2017, 12:08am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/9 "2017-10-28T00:08:31Z")

</div>

`rand()` will have a really bad conflict.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 28, 2017, 12:32am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/10 "2017-10-28T00:32:10Z")

</div>

> [@yuyichao](#):
>
> rand() will have a really bad conflict.

i wish i can understand what that means

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [October 28, 2017, 12:45am UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/11 "2017-10-28T00:45:50Z")

</div>

You are accessing the same global RNG from multiple threads. It’s a data race and can give wrong result as well as poor performance.

---

<div class="post-metadata">

**Author:** ![dalejordan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dalejordan/32/2196_2.png) [@dalejordan](https://discourse.julialang.org/u/dalejordan)\
**Post date:** [October 28, 2017, 6:32pm UTC](https://discourse.julialang.org/t/use-of-atomic-variables-in-julia/2511/12 "2017-10-28T18:32:13Z")

</div>

Checkout [RandomNumbers.jl](https://github.com/sunoru/RandomNumbers.jl), especially Random123 for cheap and cheerful threadable generators.