# Multithreading with dynamic scheduler

**URL:** <https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187>\
**Category:** Performance\
**Created:** [August 19, 2020, 12:25am UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187 "2020-08-19T00:25:31Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 12:25am UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/1 "2020-08-19T00:25:31Z")

</div>

I have an expensive function that I need to evaluate for several inputs. To use multithreading, I usually use the following workflow:

```julia
inputs = collect(deepcopy(input) for _ = 1 : Threads.nthreads())
Threads.@threads for k in iterable
	id = Threads.threadid()
	change_input!(inputs[id], k)
	expensive_function(inputs[id])
	restore_input!(inputs[id], k)
end

```

Since the time per iteration can vary, I want to use a dynamic scheduler with `@spawn`. To do so, I must ensure that a specific thread must not switch tasks before finishing the entire loop iteration. How can I do that?

It seems to me that I cannot do this without overwriting the `wait` function. Is that correct?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [August 19, 2020, 6:25am UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/2 "2020-08-19T06:25:36Z")

</div>

Could you refactor your code to use a pure function for parallel execution?  
Then you could do

```julia
f(x) = x^2 # some expensive (pure) function
inputs = 1:10
futures = [Threads.@spawn f(x) for x in inputs]
# maybe more code here which does not depend on results
results = fetch.(futures)

```

---

<div class="post-metadata">

**Author:** ![biona001](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/biona001/32/16497_2.png) [@biona001](https://discourse.julialang.org/u/biona001)\
**Post date:** [August 19, 2020, 6:36am UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/3 "2020-08-19T06:36:29Z")

</div>

You might also consider using [ThreadPools.jl](https://github.com/tro3/ThreadPools.jl) which exposes the `@qthreads` macro for a dynamically scheduled for loop

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 19, 2020, 2:27pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/4 "2020-08-19T14:27:20Z")

</div>

Maybe something like

```julia
function worker(input, channel)
    while true
        k = take!(channel)
        isnothing(k) && break
        change_input!(input, k)
        expensive_function(input)
        restore_input!(input, k)
    end
end

channel = Channel(N)
workers = []
for i = 1:N
    push!(workers, @spawn worker(deepcopy(input), channel))
end

for k in iterable
    put!(channel, k)
end

for i = 1:N
    put!(channel, nothing)
end

for worker in workers
    wait(worker)
end

```

---

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 7:34pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/5 "2020-08-19T19:34:51Z")

</div>

I thought about calling `input_copy = deepcopy(input)` inside the `for` loop and then calling the pure function `expensive_function(input_copy)`. The problem is that `input` occupies a lot of memory, so I would rather minimize the number of calls to `deepcopy`.

---

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 7:35pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/6 "2020-08-19T19:35:11Z")

</div>

I didn’t know about ThreadPools. Thanks!

---

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 7:37pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/7 "2020-08-19T19:37:16Z")

</div>

Actually, I think I cannot guarantee that each `worker` is assigned to each thread. Maybe this is something where I can use the `@tspawnat` macro of the ThreadPools package suggested by @biona001.

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [August 19, 2020, 8:01pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/8 "2020-08-19T20:01:36Z")

</div>

If your expensive\_function does not modify input (and only returns its output), there is no need to copy it.  
If it does need to modify the input, maybe there is a way to split input into separate elements (e.g. columns in a matrix) where each instance of expensive\_function only operates on a distinct part of it? Then a (distinct) view on input could be passed to each instance of the expensive\_function.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 19, 2020, 8:07pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/9 "2020-08-19T20:07:18Z")

</div>

> [@lucas711642](#):
>
> Actually, I think I cannot guarantee that each `worker` is assigned to each thread.

Why do you need that guarantee?

---

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 8:34pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/10 "2020-08-19T20:34:45Z")

</div>

For each iteration, I initially modify some of the elements of `input` (a `struct` basically composed of `Vector{Float64}` fields) and then I call `expensive_function`. Indeed, I could split `input` into two different variables where only one of them is modified in each iteration. The only problem is that it would demand much work to adapt `expensive_function` for this case.

---

<div class="post-metadata">

**Author:** ![lucas711642](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucas711642/32/12051_2.png) [@lucas711642](https://discourse.julialang.org/u/lucas711642)\
**Post date:** [August 19, 2020, 8:35pm UTC](https://discourse.julialang.org/t/multithreading-with-dynamic-scheduler/45187/11 "2020-08-19T20:35:31Z")

</div>

Indeed, I didn’t pay attention to the fact that in your code I have each `deepcopy(input)` binded to a specific task, rather than binded to a specific thread.

I’ll definitely test your solution. Thank you!