# \[ANN\] ThreadPools.jl - Improved thread management for background and nonuniform tasks

**URL:** <https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592>\
**Category:** Package Announcements\
**Created:** [January 20, 2020, 3:49pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592 "2020-01-20T15:49:29Z")\
**Posts on this page:** 11\
**Page:** 3

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [January 25, 2020, 12:31am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/41 "2020-01-25T00:31:10Z")

</div>

Update on the setup overhead above. That is only when being called in a script from global. Wrap the same script in a main() function, and the overhead goes away. Lesson learned.

![Figure_1](https://global.discourse-cdn.com/julialang/original/3X/7/0/70374d10617e03007fe7a71fde9bf95d7db6f368.png)

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [February 28, 2020, 2:26pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/42 "2020-02-28T14:26:49Z")

</div>

@tro3, very cool stuff. One thing I find myself needing is essentially `Threads.@bgspawn expr`, i.e. run this task on any thread, as long as it’s not the primary thread. How hard would it be to add something like that? Any pointers? I’m willing to take a stab at it.

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 28, 2020, 2:44pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/43 "2020-02-28T14:44:06Z")

</div>

Do you need it to be the next available thread, or any old thread? The issue is that it is difficult to do the former and avoid the primary, because the lower-level assignment mechanism can’t query the id before assignment. If one tries to write the code to resubmit after assignment to the primary, there might be a lot of cycles spent on constantly reassigning out of the primary until a “good” one hits.

If any old thread will do, then assigning to a random thread would be easy enough. But in a heavy loading case, that thread might not get to the task for while.

Which of the above sounds like your use case?

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [February 28, 2020, 2:50pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/44 "2020-02-28T14:50:54Z")

</div>

Any old thread is fine

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 28, 2020, 10:19pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/45 "2020-02-28T22:19:14Z")

</div>

Why not just `@tspawnat 2 expr` then (always sending to thread 2)? Or do you have a bunch of jobs and want to spread them out?

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [February 28, 2020, 11:11pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/46 "2020-02-28T23:11:17Z")

</div>

Here’s what I came up with by modifying the `@tspawnat` macro:

```julia
macro bgspawn(expr)
    thunk = esc(:(()->($expr)))
    var = esc(Base.sync_varname)
    quote
        local task = Task($thunk)
        task.sticky = false
        tid = rand(2:Threads.nthreads())
        ccall(:jl_set_task_tid, Cvoid, (Any, Cint), task, tid-1)
        if $(Expr(:isdefined, var))
            push!($var, task)
        end
        schedule(task)
        task
    end
end

```

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 29, 2020, 12:17am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/47 "2020-02-29T00:17:12Z")

</div>

Maybe `rand(2:Threads.nthreads())` works fine in practice. But I guess another possibility is to roll your own background worker pool?

```julia
julia> bgpool() = Channel(Inf) do request
           workers = Task[]
           for tid in 2:Threads.nthreads()
               task = @task begin
                   for (f, response) in request
                       push!(response, @async f())
                   end
               end
               task.sticky = false
               ccall(:jl_set_task_tid, Cvoid, (Any, Cint), task, tid-1)
               schedule(task)
               push!(workers, task)
           end
           Base.sync_end(workers)
       end
bgpool (generic function with 1 method)

julia> function bgspawn(f, request)
           response = Channel(1)
           try
               push!(request, (f, response))
               return take!(response)
           finally
               close(response)
           end
       end
bgspawn (generic function with 1 method)

julia> pool = bgpool()
Channel{Any}(sz_max:9223372036854775807,sz_curr:0)

julia> t = bgspawn(pool) do
           Threads.threadid()
       end
Task (runnable) @0x00007ffa541acc40

julia> fetch(t)
2

```

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 29, 2020, 1:30am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/48 "2020-02-29T01:30:27Z")

</div>

Interesting. Is `sync_end` a 1.4 thing?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 29, 2020, 2:14am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/49 "2020-02-29T02:14:43Z")

</div>

I think `Base.sync_end` has been there for a long time (I can at least find it in 0.6). But It’s an internal function which I shouldn’t have used 😅.

I don’t know a good way to do this without using it (or without relying another internal functions), though. If I were to do this without using any internals, I’d replace `Base.sync_end(workers)` with

```julia
@sync for t in workers
    @async wait(t)
end

```

Also I guess this “sync” part does not matter much as I’m not doing enough error handling. I probably should do something like

```julia
task = @task try
    for (f, response) in request
        push!(response, @async f())
    end
catch
    close(request)
    rethrow()
end

```

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [February 29, 2020, 3:50am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/50 "2020-02-29T03:50:41Z")

</div>

Ok, having done a more thorough dive into the ThreadPools code, I think what I actually want is:

```julia
using ThreadPools

pool = ThreadPools.QueuePool(2)

put!(pool) do
    println(Threads.threadid())
    # do all my cool stuff I need
end

```

So that’s great! I like `QueuePool` because it will take care of non-uniformity, which is a potential concern in my case; that is, my earlier `@bgspawn` just picked a random thread, and there’s potential that the same thread gets picked twice and the first task takes a long time and the 2nd task is stuck queued behind it. Furthermore, in my application scenario, I don’t expect all threads to be busy all the time, so making sure work is taken by non-busy threads is important as there usually will be at least one.

Anyway, very cool stuff @tro3; ThreadPools.jl is really neat.

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 29, 2020, 6:46pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592/51 "2020-02-29T18:46:33Z")

</div>

Excellent! Glad to hear it’s useful. 🙂

[Previous page](https://discourse.julialang.org/t/ann-threadpools-jl-improved-thread-management-for-background-and-nonuniform-tasks/33592.md?page=2)
