Parallelism? Async != Parallel?

Hi Guys,
for me the Documentation about Parallel Programming is kinda Confusing in Julia. For me its hard to distinguish the first two Groups of Parallel Programming.

  1. Asynchronous “tasks”, or coroutines
  2. Multi-threading

Is the first Coroutine variant only for IO things, where I have to wait for hardware or how do #1 and #2 differ? Does #1 not run in parallel but is simply “paused” & “startable” and thus only waits for things running in the background and tries to optimize this waiting time by executing other calculations in the meantime?

I would expect that if #1 would run parallel, the following parallel example would be faster than sequential. but this is not the case, can I conclude that #1 is not running parallel? or is the example just bad?

function sum_of_sum_seq(a, b)
    l = sum(a)
    r = sum(b)
    return l + r
function sum_of_sum_parallel(a, b)
    taskR = @async sum(b)
    l = sum(a)
    r = fetch(taskR)
    return l + r
a = rand(Int, 2_000_000_000)
b = rand(Int, 2_000_000_000)  
time_seq = @elapsed result_seq = sum_of_sum_seq(a, b)
time_parallel = @elapsed result_parallel = sum_of_sum_parallel(a, b)
$ JULIA_NUM_THREADS=2 julia test.jl 
  Sequential: -2088123306310354629, Time =  66.3472296 ms
  Parallel:   -2088123306310354629, Time = 100.3715951 ms

How do #1 and #2 differ? When should you use #1 or #2? Can both be scaled accross threads?

Ty in advanced

1 Like

Coroutines are concurrent, which are not necessarily parallel. For now,

To aid compatibility, code will continue to run within a single thread by default. When tasks are launched using existing primitives ( schedule , @async ), they will run only within the thread that launches them.

that’s why you got no performance gains in the example.

1 Like

just to clearify
are couroutines (#1) never parallel or is julia deciding if its worth running the couroutine in parallel ?

if its not parallel in julia, whats the point of using it?

is it just usefull if i would be waiting for IO? like asking the OS for files etc?

Coroutines run on the same thread where you start them. You will get performance benefit from @async only when using blocking operations, like IO. You have to use Threads.@spawn instead of @async to utilize more threads.


your 2nd function is still on thread 1, try

using .Threads

function sum_of_sum_parallel(a, b)
    taskR = Threads.@spawn sum(b)
    l = sum(a)
    r = fetch(taskR)
    return l + r

and you will see parallel performance.


ah okay, so to summarize
#1 for blocking operation such as IO
#2/#3 for running the code on diffrent cores/threads

or is there any function which belongs to #1 which runs on diffrent threads?

still if you work on other threads (#2) you have to use tasks (#1). As the documentation says:

Multi-threading functionality builds on tasks by allowing them to run simultaneously on more than one thread or CPU core, sharing memory.

Finally distributed computing (#3) is yet another game.


i have read that. but my thought problem is that if #1 does not run parallel and #2 is based on #1, how can #2 run parallel?
and by parallel i just mean run on multiple threads

A Task is just a task. It can run asynchronously with @async on the same thread or parallel with @spawn on parallel threads. Therefore the task mechanism is for #1 and #2.

Perhaps this is easier to understand if you think of @async and @spawn doing two things:

  1. both create a Task, then
  2. #1 starts it on the thread you are on, #2 starts it on one of all available threads.

Rob Pike had a couple of video explaining concurrency vs parallelism.


thank you very much! i think i got it!

just to see if i understood it
#1, #2 both creat Tasks, depending on how you shedule/start the task its run on the same thread or on a new one
Threads.spawn wraps the function into a Task and runs it on a diffrent thread (#2)
async also wraps the function into a Task but runs it on the same thread (#1)

#1 is concurrent
#2 is parallel

1 Like

sounds good, happy coding!

@spawn accidentally can start it on the same thread since it puts it on one of all available threads. But in my experience it chooses first other threads.