# Assigning values in threaded nested for-loop

**URL:** <https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762>\
**Category:** New to Julia\
**Tags:** parallel, multithreading, floops\
**Created:** [November 24, 2022, 5:14pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762 "2022-11-24T17:14:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![astro-kevin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/astro-kevin/32/43342_2.png) [@astro-kevin](https://discourse.julialang.org/u/astro-kevin)\
**Post date:** [November 24, 2022, 5:14pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/1 "2022-11-24T17:14:05Z")

</div>

I have some code which looks like this:

```julia
Threads.@threads for branch in tree
    Threads.@threads for twig in branch 
        append!(biglist, dostuff.(twig))
   end
end

```

where `tree` is a vector (tree) of vectors (branch) of vectors (twigs) of floats (leaves).

Obviously this isn’t thread safe, so I tried to make it threadsafe like this:

```julia
function findlength(tree)
    for branch in tree
        for twig in branch 
            biglistlength += length(twig)
        end
    end
    return biglistlength
end
biglist = zeros(findlength(tree))
j = 0
Threads.@threads for branch in tree
    Threads.@threads for twig in branch 
        for (i,leaf) in enumerate(twig)
            biglist[i+j] = dostuff(leaf)
        end
        j += length(twig)
    end
end

```

But this isn’t thread safe either because there is a race condition in the `j +=` step.

What’s the correct way of going about something like this? I looked at the threading documentation and the atomic stuff, the locking stuff, as well as the floop stuff all look to be very complicated and would almost work here. The atomic and locking seem to devolve the two loops into a serial operation which defeats the purpose of making it threaded, while the floop reduction seems to almost work except I still need to figure out the indices in my list somehow.

Edit: I know someone will tell me that nested vectors isn’t the way to go, but my real code doesn’t actually look like this, and this kind of construction is definitely what I need.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [November 24, 2022, 5:22pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/2 "2022-11-24T17:22:10Z")

</div>

Just to confirm I understood correctly: you want to flatten the tree into an array? Do you care about the order?

---

<div class="post-metadata">

**Author:** ![astro-kevin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/astro-kevin/32/43342_2.png) [@astro-kevin](https://discourse.julialang.org/u/astro-kevin)\
**Post date:** [November 24, 2022, 5:48pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/3 "2022-11-24T17:48:19Z")

</div>

Actually no, I have edited my original post to clarify what I meant. There is a function (`dostuff` which acts on the innermost element and I want to put the results in a list.

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 24, 2022, 6:21pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/4 "2022-11-24T18:21:41Z")

</div>

If you don’t insist on for loops, [Transducers.jl](https://juliafolds.github.io/Transducers.jl/dev/) provide an easy way:

```julia
biglist = tcollect(MapCat(branch -> tcollect(MapCat(twig -> dostuff.(twig)), branch)), tree)

```

You can easily test different ways to parallize, i.e., none using `collect`, threaded using `tcollect`or distributed using `dcollect`.

---

<div class="post-metadata">

**Author:** ![astro-kevin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/astro-kevin/32/43342_2.png) [@astro-kevin](https://discourse.julialang.org/u/astro-kevin)\
**Post date:** [November 25, 2022, 4:36pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/5 "2022-11-25T16:36:30Z")

</div>

Does this work if `dostuff` returns multiple outputs?

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 25, 2022, 5:40pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/6 "2022-11-25T17:40:04Z")

</div>

Sorry, had somehow missed the `append!` or read as `push!`for that matter. To collect all results into a single output, just add another `MapCat` instead of broadcasting, i.e., `twig -> collect(MapCat(dostuff), twig)`.

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 25, 2022, 6:15pm UTC](https://discourse.julialang.org/t/assigning-values-in-threaded-nested-for-loop/90762/7 "2022-11-25T18:15:54Z")

</div>

Also found that there are some updates to transducers, which allow to get rid of the inner collect calls, i.e., to avoid allocating storage here (just like your code):

```julia
tree |> MapCat(branch -> branch |> MapCat(twig -> twig |> MapCat(dostuff))) |> tcollect
tree |> MapCat(branch -> branch |> MapCat(twig -> twig |> Map(dostuff))) |> foldxt(append!!)

```

The last version using `foldxt` – like reduce but multi-threading in a tree like fashion – and `append!!`ing all results – using fast append from `BangBang` – is even more similar to your code.