# Multithreading balancing

**URL:** <https://discourse.julialang.org/t/multithreading-balancing/67220>\
**Category:** Performance\
**Tags:** multithreading\
**Created:** [August 28, 2021, 2:32pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220 "2021-08-28T14:32:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [August 28, 2021, 2:32pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/1 "2021-08-28T14:32:55Z")

</div>

Consider the following scenario (in local scope):

```julia
function ....
    @threads for i ∈ 1:128
         y[i] = dosomethingexpensive( x[i] )
    end
end

```

Say I’m running this on a machine with 32 physical cores. Then often, the last n \< 32 calls to dosomethingexpensive are completed by fewer than n threads.

What would be the best way to achieve greater balancing?

Background: an example would be the case in which each i corresponds to a replication in a simulation study, where each replication can take a few minutes, but where there is no ex ante expectation that one replication would take longer than another.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 28, 2021, 2:41pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/2 "2021-08-28T14:41:23Z")

</div>

Currently, only direct segmentation is supported by the `@threads` macro. That is, there is no work stealing API directly available yet - you’ll have to do balancing yourself. One way “around” that is to use a `Channel` of tasks, which are created ahead of time and pushed into that channel. After all tasks are created, `take!` from the channel on all threads and execute the given task, thereby emulating a work-stealing scheduler.

You can also take a look at [Threadpools.jl](https://juliahub.com/ui/Packages/ThreadPools/WlBpm/2.0.1), though that comes with some caveats due to the fact that julia doesn’t pin threads to certain CPU threads etc.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [August 28, 2021, 2:43pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/3 "2021-08-28T14:43:58Z")

</div>

FWIW, `Threads.@spawn` does load balancing (similar to `pmap` from `Distributed`).

---

<div class="post-metadata">

**Author:** ![Joris\_Pinkse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joris_pinkse/32/216398_2.png) [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Post date:** [August 28, 2021, 2:46pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/4 "2021-08-28T14:46:57Z")

</div>

Thanks.

I’ve played with something like

```julia
@sync for ....
     @spawn ...
end

```

but I recall reading on this forum that @threads is preferable for load balancing reasons… I’ll play around some more.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [August 28, 2021, 3:10pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/5 "2021-08-28T15:10:59Z")

</div>

> [@Joris\_Pinkse](#):
>
> but I recall reading on this forum that @threads is preferable for load balancing reasons… I’ll play around some more.

`@threads` has lower overhead (“is cheaper”) but **doesn’t do load balancing at all**. The iteration range of the loop is split into equals parts according to the number of available threads. OTOH, `@spawn` implements a form of load balancing but has more overhead. See [Announcing composable multi-threaded parallelism in Julia](https://julialang.org/blog/2019/07/multithreading/).

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 28, 2021, 3:27pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/6 "2021-08-28T15:27:10Z")

</div>

Yes, using `@spawn` is basically the same as managing the tasks explicitly by hand via a `Channel`. In the case of `@spawn`, it’s the julia task system that’s doing the “balancing” for you implicitly.

As of julia 1.5, `@threads` has an argument `schedule`, though currently only `:static` (“which creates one task per thread and divides the iterations equally among them”) is supported. In the future, when more kinds of scheduling would be supported, `@threads` may be the better option (though I’m not sure what the current direction of things in that regard are).

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [August 28, 2021, 10:13pm UTC](https://discourse.julialang.org/t/multithreading-balancing/67220/7 "2021-08-28T22:13:52Z")

</div>

I’ve implemented load-balancing threaded parallel loops in [FLoops.jl](https://github.com/JuliaFolds/FLoops.jl) which can also use [a wide class of scheduling policies depending on your needs](https://discourse.julialang.org/t/ann-foldsthreads-jl-a-zoo-of-pluggable-thread-based-data-parallel-execution-mechanisms/54662) (plus other things like distributed and GPU -based parallel loops and reductions).