Multithreading for recursive operations

On current master (v1.3-DEV), tasks are run on threads. Using a macro offered by Jeff Bezanson and reposted in

one can get what you describe:

right_task = @par do_stuff!(res_right, right)
do_stuff!(res_left, left)
wait(right_task)

In my experiments so far I find that there is moderate overhead and one has to be diligent about inference (function barriers are often helpful). But it works, and I see good scaling (e.g. a basic recursive task-decomposed merge-sort of Float64 on 1 core is 1.6x slower than the serial version in Base, but running on 8 cores speeds up the tasked version by 6x).

Data dependencies, task priorities, and cache-friendly algorithm design are obviously still up to the programmer (for now).

I’ve learned enough about concurrent programming to be full of admiration for the PARTR implementors, principally Kiran, Jameson, and Jeff IIUC. This is really nice infrastructure.

10 Likes