# Parallelism? Async != Parallel?

**URL:** <https://discourse.julialang.org/t/parallelism-async-parallel/45941>\
**Category:** New to Julia\
**Tags:** parallel, multithreading\
**Created:** [September 2, 2020, 10:38am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941 "2020-09-02T10:38:26Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ok\_ok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ok_ok/32/15450_2.png) [@ok\_ok](https://discourse.julialang.org/u/ok_ok)\
**Post date:** [September 2, 2020, 10:38am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/1 "2020-09-02T10:38:27Z")

</div>

Hi Guys,  
for me the Documentation about Parallel Programming is kinda Confusing in Julia. For me its hard to distinguish the first two Groups of Parallel Programming.

[https://docs.julialang.org/en/v1/manual/parallel-computing/](https://docs.julialang.org/en/v1/manual/parallel-computing/)

1. Asynchronous “tasks”, or coroutines
2. Multi-threading

Is the first Coroutine variant only for IO things, where I have to wait for hardware or how do #1 and #2 differ? Does #1 not run in parallel but is simply “paused” & “startable” and thus only waits for things running in the background and tries to optimize this waiting time by executing other calculations in the meantime?

I would expect that if #1 would run parallel, the following parallel example would be faster than sequential. but this is not the case, can I conclude that #1 is not running parallel? or is the example just bad?

```julia
function sum_of_sum_seq(a, b)
    l = sum(a)
    r = sum(b)
    return l + r
end  
function sum_of_sum_parallel(a, b)
    taskR = @async sum(b)
    l = sum(a)
    r = fetch(taskR)
    return l + r
end  
a = rand(Int, 2_000_000_000)
b = rand(Int, 2_000_000_000)  
time_seq = @elapsed result_seq = sum_of_sum_seq(a, b)
time_parallel = @elapsed result_parallel = sum_of_sum_parallel(a, b)
  
$ JULIA_NUM_THREADS=2 julia test.jl 
  Sequential: -2088123306310354629, Time = 66.3472296 ms
  Parallel: -2088123306310354629, Time = 100.3715951 ms

```

How do #1 and #2 differ? When should you use #1 or #2? Can both be scaled accross threads?

Ty in advanced

---

<div class="post-metadata">

**Author:** ![Gnimuc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gnimuc/32/2194_2.png) [@Gnimuc](https://discourse.julialang.org/u/Gnimuc)\
**Post date:** [September 2, 2020, 10:48am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/2 "2020-09-02T10:48:01Z")

</div>

Coroutines are concurrent, which are not necessarily parallel. For now,

> [To aid compatibility, code will continue to run within a single thread by default. When tasks are launched using existing primitives ( `schedule` , `@async` ), they will run only within the thread that launches them.](https://julialang.org/blog/2019/07/multithreading/#task_scheduling_and_synchronization)

that’s why you got no performance gains in the example.

---

<div class="post-metadata">

**Author:** ![ok\_ok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ok_ok/32/15450_2.png) [@ok\_ok](https://discourse.julialang.org/u/ok_ok)\
**Post date:** [September 2, 2020, 10:50am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/3 "2020-09-02T10:50:42Z")

</div>

just to clearify  
are couroutines (#1) never parallel or is julia deciding if its worth running the couroutine in parallel ?

if its not parallel in julia, whats the point of using it?

is it just usefull if i would be waiting for IO? like asking the OS for files etc?

---

<div class="post-metadata">

**Author:** ![tisztamo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tisztamo/32/16200_2.png) [@tisztamo](https://discourse.julialang.org/u/tisztamo)\
**Post date:** [September 2, 2020, 10:59am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/4 "2020-09-02T10:59:44Z")

</div>

Coroutines run on the same thread where you start them. You will get performance benefit from `@async` only when using blocking operations, like IO. You have to use `Threads.@spawn` instead of `@async` to utilize more threads.

---

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [September 2, 2020, 11:01am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/5 "2020-09-02T11:01:37Z")

</div>

your 2nd function is still on thread 1, try

```julia
using .Threads

function sum_of_sum_parallel(a, b)
    taskR = Threads.@spawn sum(b)
    l = sum(a)
    r = fetch(taskR)
    return l + r
end

```

and you will see parallel performance.

---

<div class="post-metadata">

**Author:** ![ok\_ok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ok_ok/32/15450_2.png) [@ok\_ok](https://discourse.julialang.org/u/ok_ok)\
**Post date:** [September 2, 2020, 11:03am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/6 "2020-09-02T11:03:41Z")

</div>

ah okay, so to summarize  
#1 for blocking operation such as IO  
#2/#3 for running the code on diffrent cores/threads

or is there any function which belongs to #1 which runs on diffrent threads?

---

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [September 2, 2020, 11:11am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/7 "2020-09-02T11:11:57Z")

</div>

still if you work on other threads (#2) you have to use tasks (#1). As the [documentation](https://docs.julialang.org/en/v1/manual/parallel-computing/) says:

> Multi-threading functionality builds on tasks by allowing them to run simultaneously on more than one thread or CPU core, sharing memory.

Finally distributed computing (#3) is yet another game.

---

<div class="post-metadata">

**Author:** ![ok\_ok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ok_ok/32/15450_2.png) [@ok\_ok](https://discourse.julialang.org/u/ok_ok)\
**Post date:** [September 2, 2020, 11:20am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/8 "2020-09-02T11:20:10Z")

</div>

i have read that. but my thought problem is that if #1 does not run parallel and #2 is based on #1, how can #2 run parallel?  
and by parallel i just mean run on multiple threads

---

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [September 2, 2020, 11:31am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/9 "2020-09-02T11:31:03Z")

</div>

A `Task` is just a task. It can run asynchronously with `@async` on the same thread or parallel with `@spawn` on parallel threads. Therefore the task mechanism is for #1 and #2.

Perhaps this is easier to understand if you think of `@async` and `@spawn` doing two things:

1. both create a [`Task`](https://docs.julialang.org/en/v1/base/parallel/#Core.Task), then
2. #1 starts it on the thread you are on, #2 starts it on one of all available threads.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 2, 2020, 11:38am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/10 "2020-09-02T11:38:40Z")

</div>

Rob Pike had a couple of video explaining concurrency vs parallelism.

---

<div class="post-metadata">

**Author:** ![ok\_ok](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ok_ok/32/15450_2.png) [@ok\_ok](https://discourse.julialang.org/u/ok_ok)\
**Post date:** [September 2, 2020, 11:45am UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/11 "2020-09-02T11:45:21Z")

</div>

thank you very much! i think i got it!

just to see if i understood it  
#1, #2 both creat Tasks, depending on how you shedule/start the task its run on the same thread or on a new one  
example:  
Threads.spawn wraps the function into a Task and runs it on a diffrent thread (#2)  
async also wraps the function into a Task but runs it on the same thread (#1)

#1 is concurrent  
#2 is parallel

---

<div class="post-metadata">

**Author:** ![pbayer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbayer/32/11675_2.png) [@pbayer](https://discourse.julialang.org/u/pbayer)\
**Post date:** [September 2, 2020, 12:00pm UTC](https://discourse.julialang.org/t/parallelism-async-parallel/45941/12 "2020-09-02T12:00:41Z")

</div>

sounds good, happy coding!

`@spawn` accidentally can start it on the same thread since it puts it on one of all available threads. But in my experience it chooses first other threads.