# Multi-threading appears to be single thread when some threads cost much more time than the others?
**URL:** https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353
**Category:** General Usage
**Tags:** question
**Created:** [July 8, 2023, 4:13pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353 "2023-07-08T16:13:24Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@WellWellww](https://discourse.julialang.org/u/WellWellww)
#### Post date: [July 8, 2023, 4:13pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/1 "2023-07-08T16:13:24Z")
Hi, I have the following code section with millions of loops using multi-threading.
```julia
global result_list = Vector{Union{Nothing,SparseVector{Float64,Int64}}}(nothing, tot_num)
Threads.@threads for element in element_list
ind = element_to_ind_Dict[element] # get the index
@show ind
result = function_of_element(element) # the main calculation
if result === nothing
continue
end
@show result
global result_list
result_list[ind] = result
end
```
I know that in some loops the `function_of_element() ` costs much longer time than the others. Although there are millions of loops waiting for being calculated, it still looks like I am using only one thread for most of the time.
The `htop` gives the following
where the main thread is at the status of Interruptible Sleep.
Is there any method to avoid this and fully use the resource? Thanks a lot!
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [July 8, 2023, 6:37pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/2 "2023-07-08T18:37:57Z")
Did you start Julia with several threads? What do you see when you run `Threads.nthreads()`?
---
### Author:  [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)
#### Post date: [July 8, 2023, 7:08pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/3 "2023-07-08T19:08:15Z")
`@threads for` simply divides the serially ordered work into contiguous chunks and assigns those chunks to `nthreads()` tasks. So if chunk N has most of the big iterates, its task will keep running long after the others are done. A fairer alternative is to start up `nthreads()` worker tasks which `take` work items from a `Channel`. If you know in advance which iterates are most costly, they should probably go into the queue first.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [July 8, 2023, 7:34pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/4 "2023-07-08T19:34:19Z")
I thought in recent Julia versions the thread scheduling was dynamic?
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [July 8, 2023, 9:05pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/5 "2023-07-08T21:05:54Z")
dynamic != load balancing. We don’t have a load balancing scheduling option for `@threads` yet.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [July 8, 2023, 9:07pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/6 "2023-07-08T21:07:40Z")
Can you elaborate on the difference?
> [@WellWellww](#):
>
> Although there are millions of loops waiting for being calculated, it still looks like I am using only one thread for most of the time.
If this is true, why are the other threads idle even though there is yet work to be done?
---
### Author:  [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)
#### Post date: [July 8, 2023, 9:08pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/7 "2023-07-08T21:08:20Z")
“Dynamic scheduling” in this context means that once a task has been defined, different portions may execute on different threads (sequentially, unless it has sub-tasks). The logic for _building_ tasks in base/threadingconstructs.jl seems to have been the same for quite a while.
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [July 8, 2023, 9:17pm UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/8 "2023-07-08T21:17:25Z")
Is there a reference in the docs to understand this a little better? My quick search yielded nothing
---
### Author:  [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)
#### Post date: [July 9, 2023, 1:31am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/9 "2023-07-09T01:31:03Z")
The current situation is apparently an “implementation detail”, and therefore (!) not well documented. The manual explicitly says that the `@threads` body “must not make any assumptions about the distribution of iterations to tasks …,” which leads some of us to prefer other constructs.
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [July 9, 2023, 2:02am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/10 "2023-07-09T02:02:21Z")
```julia
using ThreadsX
function calc_result_list(element_list, element_to_ind_Dict, tot_num)
result_list = Vector{Union{Nothing,SparseVector{Float64,Int64}}}(nothing, tot_num)
ThreadsX.foreach(element_list, basesize=1) do element
ind = element_to_ind_Dict[element] # get the index
@show ind
result = function_of_element(element) # the main calculation
if result === nothing
return
end
@show result
result_list[ind] = result
end
return result_list
end
```
`ThreadsX` allows you to pass a `basesize`. Using a `basesize=1` means tasks work on 1 iteration at a time, rather than large chunks.
The above implementation also avoids globals.
---
### Author:  [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)
#### Post date: [July 9, 2023, 2:58am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/11 "2023-07-09T02:58:02Z")
By the way, if you really have many iterations, you should drop the `@show` lines which block for access to the IO event loop.
---
### Author:  [@WellWellww](https://discourse.julialang.org/u/WellWellww)
#### Post date: [July 9, 2023, 11:23am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/14 "2023-07-09T11:23:28Z")
> [@Ralph\_Smith](#):
>
> which leads some of us to prefer other constructs.
Hi, thanks a lot for your explanations. May I ask what the “other constructs” are?
---
### Author:  [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)
#### Post date: [July 10, 2023, 3:36am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/15 "2023-07-10T03:36:53Z")
Re: other constructs
In addition to Chris Elrod’s example above, there are some good ones in the Transducers/FLoops packages and no end of (potentially dangerous) things one can do oneself with `@spawn`.
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [July 10, 2023, 9:53am UTC](https://discourse.julialang.org/t/multi-threading-appears-to-be-single-thread-when-some-threads-cost-much-more-time-than-the-others/101353/16 "2023-07-10T09:53:45Z")
> [@gdalle](#):
>
> Can you elaborate on the difference?
The task structure of `@threads :dynamic` and `@threads :static` is actually the same, i.e. both create O(nthreads()) tasks corresponding to contiguous regions of the given (potentially large) iteration range. The only difference is that tasks can migrate (“are non-sticky”) in the former case while they can’t in the latter. Compare this to `@sync for ... @spawn ...` which create one task per loop iteration and gives a form of load-balancing through Julias task scheduler. In pictures:
Note that neither `:static` nor `:dynamic` gives load balancing (as `@spawn` does). Also note that the task-\>thread mapping isn’t fixed for `:dynamic` but is for `:static`. However, when we sort by workload we see that eventually they do the same thing. So, to summarize, which Julia thread does which chunk is dynamically decided of `:dynamic` but the chunks are the same as for `:static`.
(Pluto notebook: [load\_balancing.jl](https://discourse.julialang.org/uploads/short-url/uMwOcFRfq9fqknrL2DSan36HZ3l.jl) (45.8 KB) - Be aware though that I use hacky/unsafe `threadid()` pattern here for simplicity.)
You might want to check out these comments by @tkf:
- [Behavior of `Threads.@threads for` loop - #17 by tkf](https://discourse.julialang.org/t/behavior-of-threads-threads-for-loop/76042/17)
- [Feature request: a work stealing threaded for loop · Issue #21017 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/21017#issuecomment-1049663002)
and the comments in these PRs:
- [Clarify the behavior of `@threads for` by tkf · Pull Request #44168 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/44168)
- [https://github.com/JuliaLang/julia/pull/43919](https://github.com/JuliaLang/julia/pull/43919)
- [Feature request: a work stealing threaded for loop · Issue #21017 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/21017)