# Multi-threading appears to be single thread when some threads cost much more time than the others?

**URL:** https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353
**Category:** General Usage
**Tags:** question
**Created:** [July 8, 2023, 4:13pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353 "2023-07-08T16:13:24Z")
**Posts on this page:** 1
**Showing post:** 16

<div class="post-metadata">

### Author: ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [July 10, 2023, 9:53am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/16 "2023-07-10T09:53:45Z")

</div>

> [@gdalle](#):
>
> Can you elaborate on the difference?

The task structure of `@threads :dynamic` and `@threads :static` is actually the same, i.e. both create O(nthreads()) tasks corresponding to contiguous regions of the given (potentially large) iteration range. The only difference is that tasks can migrate (“are non-sticky”) in the former case while they can’t in the latter. Compare this to `@sync for ... @spawn ...` which create one task per loop iteration and gives a form of load-balancing through Julias task scheduler. In pictures:

 ![load_balancing](https://global.discourse-cdn.com/julialang/original/3X/f/0/f0100a0556568646d4d2c5a134a5b9f2c4a34d4c.png)

 ![load_sorted](https://global.discourse-cdn.com/julialang/original/3X/4/e/4e4810f8aa5bdbe7cbddf7bf9dcd055b6cb0fbe7.png)

Note that neither `:static` nor `:dynamic` gives load balancing (as `@spawn` does). Also note that the task-\>thread mapping isn’t fixed for `:dynamic` but is for `:static`. However, when we sort by workload we see that eventually they do the same thing. So, to summarize, which Julia thread does which chunk is dynamically decided of `:dynamic` but the chunks are the same as for `:static`.

(Pluto notebook: [load\_balancing.jl](https://discourse.julialang.org/uploads/short-url/uMwOcFRfq9fqknrL2DSan36HZ3l.jl) (45.8 KB) - Be aware though that I use hacky/unsafe `threadid()` pattern here for simplicity.)

You might want to check out these comments by @tkf:

- [Behavior of `Threads.@threads for` loop - #17 by tkf](https://discourse.julialang.org/t/behavior-of-threads-threads-for-loop/76042/17)
- [Feature request: a work stealing threaded for loop · Issue #21017 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/21017#issuecomment-1049663002)

and the comments in these PRs:

- [Clarify the behavior of `@threads for` by tkf · Pull Request #44168 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/44168)
- [https://github.com/JuliaLang/julia/pull/43919](https://github.com/JuliaLang/julia/pull/43919)
- [Feature request: a work stealing threaded for loop · Issue #21017 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/21017)

---

_[View the full topic](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353)._