Threading, Threads.@threads vs polyester.@batch vs LoopVectorization.@tturbo

@Elrod is the mastermind behind these tools, so he can better answer how they evolved, but here is a quick description of how I use them.

@tturbo is threading+SIMD instructions (CPU instructions that act simultaneously on 4 or 8 neighboring array elements). It is just a threaded version of @turbo and it uses Polyester for the threading. It is meant for parallelizing simple inner loops, which typically are operations where a single execution of the bare loop takes no more than a couple hundred nanoseconds. You would probably never use @tturbo on something that is not an array of isbits objects.

Polyester.@batch hijacks the threads provided by Julia and uses a much faster, but simpler scheduler. It simply does not provide as many ways to nest threads as @threads. Because of its simplicity it has drastically lower overhead, so it is useful for multi-threading things that are already very fast (while setting up the @threads scheduling might be slower than your fast operation). Usually you should use @batch only if your threaded jobs might often be small.

You can nest @batch inside of @thread but the scheduling of the threads might get very confused. I usually just disable the Polyester threads when I do such nesting. I think the documentation (and accompanying benchmarks) of this thread-disabling feature (implemented 2 days ago) would be of interest to you: https://github.com/JuliaSIMD/Polyester.jl#disabling-polyester-threads (at the bottom of the README)

8 Likes