# How to add arrays using multithreading?

**URL:** <https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532>\
**Category:** General Usage\
**Tags:** question\
**Created:** [March 6, 2018, 10:40am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532 "2018-03-06T10:40:38Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 10:40am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/1 "2018-03-06T10:40:38Z")

</div>

I have a prototype program looking like

```julia
function calculate(i)
    # do something, maybe expensive
    return eye(2)*rand()
end

s = zeros(2, 2)
Thread.@thread for i in 1:100 
    s += calculate(i)
end

```

I understand this program is not thread-safe since multiple threads may be accessing `s` at the same time. However, since the document of multithreading is very short. I don’t know how to fix the problem. Anyone offer some help?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 6, 2018, 10:46am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/2 "2018-03-06T10:46:53Z")

</div>

Perhaps something like

```julia
function calculate(i)
    return sin(i)
end

s = zeros(Threads.nthreads())
Threads.@threads for i in 1:10^6
    s[Threads.threadid()] += calculate(i)
end
sum(s)

```

each thread write to their own part and then you do a serial reduction in the end.

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 11:10am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/3 "2018-03-06T11:10:16Z")

</div>

Thank you. This definitely works. However, this solution is effectively storing everything `calculate` returns, which costs a lot of memory in my case. Is there anything like lock and unlock?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [March 6, 2018, 11:13am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/4 "2018-03-06T11:13:44Z")

</div>

> [@Chong\_Wang](#):
>
> this solution is effectively storing everything calculate returns,

pretty sure it only stores as many variabels as there are threads, so negligible RAM usage.

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 11:14am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/5 "2018-03-06T11:14:47Z")

</div>

Good catch! I misunderstood @kristoffer.carlsson. Thank you!

---

<div class="post-metadata">

**Author:** ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)\
**Post date:** [March 6, 2018, 11:27am UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/6 "2018-03-06T11:27:47Z")

</div>

Using locks/unlocks:

```julia
function calculate(i)
    # do something, maybe expensive
    return eye(2)*rand()
end

s = zeros(2, 2)
mutex = Threads.Mutex()
Thread.@thread for i in 1:100 
    Threads.lock(mutex)
    s += calculate(i)
    Threads.unlock(mutex)
end

```

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [March 6, 2018, 12:05pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/7 "2018-03-06T12:05:37Z")

</div>

That would have terrible performance, since all threads are waiting and only one is working at any one time, so it’s a single-thread working disguised in multi-threading. If `calculate` is taken out of the lock and is really expensive then it will be better, but then the intermediate still needs to be stored, so you might as well avoid waiting altogether and use Kristoffer’s solution.

Edit: if it was just a demo of locks, then I may have over-reacted 🙂

---

<div class="post-metadata">

**Author:** ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)\
**Post date:** [March 6, 2018, 12:34pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/8 "2018-03-06T12:34:10Z")

</div>

it was 😉

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 12:34pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/9 "2018-03-06T12:34:54Z")

</div>

> [@louisponet](#):
>
> function calculate(i)  
> # do something, maybe expensive  
> return eye(2)\*rand()  
> end
> 
> s = zeros(2, 2)  
> mutex = Threads.Mutex()  
> Thread.@thread for i in 1:100  
> Threads.lock(mutex)  
> s += calculate(i)  
> Threads.unlock(mutex)  
> end

Multithreading is new to me. A simple question: If I do

```julia
function calculate(i)
    # do something, maybe expensive
    return eye(2)*rand()
end

s = zeros(2, 2)
mutex = Threads.Mutex()
Thread.@thread for i in 1:100 
    result = calculate(i)
    Threads.lock(mutex)
    s += result
    Threads.unlock(mutex)
end

```

Will there be any conflicts for name bindings of `result`?

---

<div class="post-metadata">

**Author:** ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)\
**Post date:** [March 6, 2018, 12:35pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/10 "2018-03-06T12:35:46Z")

</div>

No, all the variables that are created inside the @threads block will be private to each thread.

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 12:39pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/11 "2018-03-06T12:39:10Z")

</div>

Also, due to [https://github.com/JuliaLang/julia/issues/22581](https://github.com/JuliaLang/julia/issues/22581), do I need to avoid matrix mulplication in `calculate`?

---

<div class="post-metadata">

**Author:** ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)\
**Post date:** [March 6, 2018, 12:43pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/12 "2018-03-06T12:43:53Z")

</div>

Interesting, this is an issue I haven’t personally ran into. I’ve been doing excessive amounts of matrix multiplications inside loops, but probably not exactly in the way that the issue states. I guess just avoid this situation?

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [March 6, 2018, 12:54pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/13 "2018-03-06T12:54:02Z")

</div>

Maybe. Although I cannot identify the situation confidently and I need to do quite a lot of things inside `calculate`. Maybe that’s why multithreading is labeled as experimental in the document.

---

<div class="post-metadata">

**Author:** ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)\
**Post date:** [March 6, 2018, 1:46pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/14 "2018-03-06T13:46:34Z")

</div>

Best way to check these things, is to just run the calculation a couple of times and check that the results are always the same.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [March 6, 2018, 2:45pm UTC](https://discourse.julialang.org/t/how-to-add-arrays-using-multithreading/9532/15 "2018-03-06T14:45:21Z")

</div>

There is another problem with this code as `rand()` is not thread safe. See end of section [https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Threading-(Experimental)-1](https://docs.julialang.org/en/latest/manual/parallel-computing/#Multi-Threading-(Experimental)-1) in the Julia Manual.