I am happy to announce the registration of ThreadPools.jl.
As of Julia 1.3, the algorithm of the @threads macro is to pre-divide the incoming range into equal partitions running on each thread. This is very efficient for most use cases, but there are a couple of scenarios where this is a problem:
- When something else is running on the primary thread, like a GUI or a web server
- When the tasks are very nonuniform - this can lead to one thread working on the longer tasks while the others sit idle
ThreadPools.jl exposes @fgthreads and @bgthreads, both of which actively manage the thread pool so that when one task finishes, the next one in the queue starts. If one thread is hit with a long task, the others will keep processing the shorter tasks and if a second long one comes along, it will be processed in parallel with the first long one.
I had to produce this package for myself, as I hit both of the above pain points at the same time in my use case. This community has been very helpful, so I figured Iâd fill out the corners, document it, and see if you all find it useful. If so, I am committed to keeping it maintained for as long as needed. This is my first registered package, so if there is anything I missed in making this more useful, donât hesitate to let me know.
Usage
Basic usage is pretty easy. Just replace @threads or map with @fgthreads/@bgthreads or fgmap/bgmap (bg versions are background, keeping the primary thread free):
julia> @bgthreads for x in 1:3
println("$x $(Threads.threadid())")
end
2 3
3 4
1 2
julia> bgmap([1,2,3]) do x
println("$x $(Threads.threadid())")
x^2
end
2 3
3 4
1 2
3-element Array{Int64,1}:
1
4
9
Logging
I found this very useful. There are logging versions of all the functions to help tune performance:
julia> ThreadPools.logfgforeach(x -> sleep(0.1*x), "log.txt", 1:8)
julia> ThreadPools.showstats("log.txt")
Total duration: 1.217 s
Number of jobs: 8
Average job duration: 0.46 s
Minimum job duration: 0.112 s
Maximum job duration: 0.805 s
Thread 1: Duration 1.217 s, Gap time 0.0 s
Thread 2: Duration 0.827 s, Gap time 0.0 s
Thread 3: Duration 0.613 s, Gap time 0.0 s
Thread 4: Duration 1.023 s, Gap time 0.0 s
julia> ThreadPools.showactivity("log.txt", 0.1)
0.000 - - - -
0.100 4 2 1 3
0.200 4 2 5 3
0.300 4 6 5 3
0.400 4 6 5 7
0.500 8 6 5 7
0.600 8 6 5 7
0.700 8 6 - 7
0.800 8 6 - 7
0.900 8 - - 7
1.000 8 - - 7
1.100 8 - - -
1.200 8 - - -
1.300 - - - -
1.400 - - - -
There is also a logging version of the original @threads, so you can measure performance of your use case with the built-in macro.
Structure
The ThreadPool is simply a Channel of Tasks that feeds to a number of Task handlers on each thread. Each handler processes the task and hands back to an output Channel. Then there is a result iterator to fetch the results:
put!(pool, fn, args...)
|
V
Channel{Task}
|
|---------------------------- ...
| |
V V
Thread 1 Handler Thread 2 Handler ...
(optional) |
| |
|<--------------------------- ...
|
V
Channel{Task}
|
V
results(pool)
Pretty straightforward stuff - the channels serve as a queue of tasks to be completed, and the handlers just process them as they become available. I attempted to wrap it into an even simpler API so the under-the-hood stuff doesnât have to be dealt with.
Anyhow, thanks for your help, and all feedback appreciated. Cheers!
