# \[ANN\] ThreadPools.jl v1.0.0 - Improved thread management for background and nonuniform tasks

**URL:** <https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138>\
**Category:** Package Announcements\
**Created:** [February 3, 2020, 11:07pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138 "2020-02-03T23:07:18Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 3, 2020, 11:07pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/1 "2020-02-03T23:07:18Z")

</div>

All,

I’m happy to announce a (breaking) API upgrade to [ThreadPools.jl](https://github.com/tro3/ThreadPools.jl) to better separate choices around handling background tasks and handling nonuniform tasks. Many thanks to @tkf and @ianshmean and many others who contributed to the ideas.

### Overview

ThreadPools.jl is a simple package for thread parallelization that exposes a few macros and functions that mimic `Base.Threads.@threads` , `Base.map` , and `Base.foreach`. These macros/functions handle cases that the built-in functions are not always well-suited for:

- Groups of tasks that the user wants to keep off of the primary thread
- Groups of tasks that are very nonuniform in duration

For the first case, ThreadPools exposes a `@bthreads` (“background threads”) macro that behaves identically to `Threads.@threads` , but keeps the primary thread job-free. For the second case, the package exposes a `@qthreads` (“queued threads”) macro. This macro queues up the jobs before thread assignment, only starting a new job when a job on any previous thread has finished. The background version of the same is the `@qbthreads` (“queued background threads”) macro. Versions of `map` and `foreach` with the same prefixes are also available:

| | Foreground (primary allowed) | Background (primary forbidden) |
| --- | --- | --- |
| **Uniform tasks** | 
- `Base.Threads.@threads`
- `ThreadPools.pmap(fn, itrs)`
- `ThreadPools.pforeach(fn, itrs)`

 | 
- `ThreadPools.@bthreads`
- `ThreadPools.bmap(fn, itrs)`
- `ThreadPools.bforeach(fn, itrs)`

 |
| **Nonuniform tasks** | 
- `ThreadPools.@qthreads`
- `ThreadPools.qmap(fn, itrs)`
- `ThreadPools.qforeach(fn, itrs)`

 | 
- `ThreadPools.@qbthreads`
- `ThreadPools.qbmap(fn, itrs)`
- `ThreadPools.qbforeach(fn, itrs)`

 |

The package also exposes a lower-level `@pspawnat` macro that mimics the `Base.Threads.@spawn` macro, but allows direct thread assignment for users who want to develop their own scheduling.

### Logging

There are also logging versions of each of the above:

```julia-auto
julia> using Plots

pool = logpforeach(x -> sleep(0.01*x), 1:64);

plot(pool)

```

 ![Figure_1](https://global.discourse-cdn.com/julialang/original/3X/d/7/d78dfe6cf2f5d1d543c0d996d556d0101addc6c0.png)

```julia-auto
pool = logqforeach(x -> sleep(0.01*x), 1:64);

plot(pool)

```

 ![Figure_2](https://global.discourse-cdn.com/julialang/original/3X/3/d/3d30b2a90a185b2c496835683856c2de26b500ab.png)

Note that the above examples demonstrate the two scheduling strategies far better than I can describe. 😉

### Composing

There may be times where the default macros don’t quite do what you want. For those cases, a composable API is exposed. For example, perhaps we want to limit usage to only two threads off of the primary:

```julia-auto
julia> pwith(ThreadPools.QueuePool(2,2)) do pool
         pforeach(pool, x -> println((x,Threads.threadid())), 1:8)
       end;
(2, 2)
(1, 3)
(3, 2)
(4, 3)
(5, 2)
(6, 3)
(7, 2)
(8, 3)

```

Here, only threads 2 and 3 are used (in this case, on an 8-thread machine).

Anyway, thanks for all the early-adopter inputs, and I hope that some find the package useful. Feel free to submit issues at [ThreadPools.jl](https://github.com/tro3/ThreadPools.jl) if you find problems or have requests. Cheers, all!

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [February 3, 2020, 11:11pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/2 "2020-02-03T23:11:52Z")

</div>

This is great. I had a need for a thread-specific spawn-like function, so really appreciate the addition of `@pspawnat`

```julia
julia> using ThreadPools
julia> t = @pspawnat 4 Threads.threadid()
Task (runnable) @0x0000000010743c70

julia> fetch(t)
4

```

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 3, 2020, 11:14pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/3 "2020-02-03T23:14:10Z")

</div>

@oxinabox just pointed out (correctly) that `pmap` overloads the standard lib `Distributed.pmap`, which I’d really prefer not to do. Sigh. I may have to change those functions to `tmap` and `tforeach`.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 3, 2020, 11:39pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/4 "2020-02-03T23:39:11Z")

</div>

Congratulations for v1 release. The logging plots are beautiful!

Also, I can’t help but thinking to support `Distributed`+`Base.Threads` pool on top of this interface. It would be amazing if we can use multiple threads of multiple machines by just `pmap(fn, DistributedThreadPool(), itr)`…

Re:

> [@tro3](#):
>
> `pforeach(pool, x -> println((x,Threads.threadid())), 1:8)`

I think it may be better to remove the restriction `::Function` from `pforeach(fn::Function, pool, itr)` and deprecating `pforeach(pool, fn::Function, itr)`. Please see [Remove ::Function restrictions? · Issue #6 · tro3/ThreadPools.jl · GitHub](https://github.com/tro3/ThreadPools.jl/issues/6). (I’m sorry to start nitpicking the API right off the bat…)

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 3, 2020, 11:41pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/5 "2020-02-03T23:41:04Z")

</div>

@tkf, no problem. The point is to make this thing useful, and you all have way more context on how to do that than I do. 🙂

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [February 4, 2020, 3:26am UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/6 "2020-02-04T03:26:37Z")

</div>

```julia
(v1.3) pkg> add ThreadPools
 Resolving package versions...
 Installed ThreadPools ─ v0.2.0
  Updating `~/.julia/environments/v1.3/Project.toml`
  [b189fb0b] + ThreadPools v0.2.0
  Updating `~/.julia/environments/v1.3/Manifest.toml`
  [b189fb0b] + ThreadPools v0.2.0

```

Well, I guess 1.0.0 is not out yet.

I spoke too soon!

```julia
(v1.3) pkg> up
  Resolving package versions...
 Installed ThreadPools ─ v1.0.0
  Updating `~/.julia/environments/v1.3/Project.toml`
  [b189fb0b] ↑ ThreadPools v0.2.0 ⇒ v1.0.0
  Updating `~/.julia/environments/v1.3/Manifest.toml`
  [b189fb0b] ↑ ThreadPools v0.2.0 ⇒ v1.0.0

```

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 5, 2020, 5:50pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/7 "2020-02-05T17:50:49Z")

</div>

@StevenSiew is it working okay for you now?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [February 5, 2020, 6:57pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/8 "2020-02-05T18:57:59Z")

</div>

> [@tro3](#):
>
> @oxinabox just pointed out (correctly) that `pmap` overloads the standard lib `Distributed.pmap` , which I’d really prefer not to do. Sigh. I may have to change those functions to `tmap` and `tforeach` .

IIRC, `Distributed.pmap` does load balancing while your `ThreadPools.pmap` does not. Another reason why it should perhaps have a different name.

Otherwise, great work! Some of this, like for example `@pspawnat`, should go into Base at some point, IMHO.

---

<div class="post-metadata">

**Author:** ![tro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tro3/32/12355_2.png) [@tro3](https://discourse.julialang.org/u/tro3)\
**Post date:** [February 5, 2020, 10:25pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/9 "2020-02-05T22:25:38Z")

</div>

> [@carstenbauer](#):
>
> Another reason why it should perhaps have a different name.

Completely agreed. I was going to go with the “use `tmap` and deprecate `pmap`”, suggestion in 1.0.1. Then the question becomes whether to propagate the prefix to `tforeach` and `@tspawnat` as well, for consistency.

> [@carstenbauer](#):
>
> Otherwise, great work! Some of this, like for example `@pspawnat` , should go into Base at some point, IMHO.

Thanks, and agreed (in that order). 🙂

---

<div class="post-metadata">

**Author:** ![Chamika\_Kasun](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chamika_kasun/32/202926_2.png) [@Chamika\_Kasun](https://discourse.julialang.org/u/Chamika_Kasun)\
**Post date:** [November 28, 2023, 12:51pm UTC](https://discourse.julialang.org/t/ann-threadpools-jl-v1-0-0-improved-thread-management-for-background-and-nonuniform-tasks/34138/10 "2023-11-28T12:51:39Z")

</div>

> [@ianshmean](#):
>
> awn-like function, so really appreciate the addition of `@pspawnat`
> 
> ```julia
> 
> ```

@tro3 , Is it recommended to use ThreadPools with I/O heavy tasks, for something like below ?

```julia
function simulate_api_call()
     println("API call started with Thread : $(Threads.threadid())")

     # Simulate a long-running API call
     # response_task = @async HTTP.get("https://randomuser.me/api/?results=50000")

     response_task = Threads.@spawn HTTP.get("https://randomuser.me/api/?results=50000")

     println("API call completed with Thread : $(Threads.threadid())")

     return response_task
 end

function test_spawn_3()
    results = Vector{Task}(undef, Threads.nthreads() - 1)

    ThreadPools.logqbforeach(1:Threads.nthreads() - 1) do i
        results[i] = simulate_api_call()
    end

    responses = fetch.(results)

    return responses
end

```