Multi-threaded worker processes

Related:

https://github.com/JuliaLang/julia/issues/17887

FYI, as Transducers.jl has thread- and Distributed.jl-based parallelisms with a common interface (ref: Thread- and process-based parallelisms in Transducers.jl (+ some news)), it should not be too hard to stitch things together to use thread-based reduction inside Distributed.jl-based reduction. This would then automatically give us a superset of pmap that supports other processings like filtering to be fused with mapping.

5 Likes