# Using Tasks 101

**URL:** <https://discourse.julialang.org/t/using-tasks-101/114878>\
**Category:** Teaching & Outreach\
**Tags:** tutorials, task\
**Created:** [May 28, 2024, 9:31pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878 "2024-05-28T21:31:45Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [May 28, 2024, 9:31pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/1 "2024-05-28T21:31:45Z")

</div>

I’m not sure if this is the right place to post this, but I spent a lot of time learning about Tasks recently for a project at work, and wrote up a blog post tutorial summarizing how to use them: [https://www.lesswrong.com/posts/kPnjPfp2ZMMYfErLJ/julia-tasks-101](https://www.lesswrong.com/posts/kPnjPfp2ZMMYfErLJ/julia-tasks-101)

Feedback and corrections appreciated!

---

<div class="post-metadata">

**Author:** ![Sevi](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@Sevi](https://discourse.julialang.org/u/Sevi)\
**Post date:** [May 31, 2024, 7:23am UTC](https://discourse.julialang.org/t/using-tasks-101/114878/2 "2024-05-31T07:23:47Z")

</div>

I really appreciate this post! Especially as someone who needed several attempts to wrap my head around both the concepts and the nomenclature of tasks/threads/coroutines/… (in general, not particularly in Julia).

Given that the reader already knows about the existence of hardware/OS threads (which I think is a given if one is looking for this kind of information), it’s straight to the point and very accessible 🙂

---

<div class="post-metadata">

**Author:** ![Gast](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gast/32/206425_2.png) [@Gast](https://discourse.julialang.org/u/Gast)\
**Post date:** [July 21, 2024, 5:31pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/3 "2024-07-21T17:31:07Z")

</div>

I liked your post, very useful! Thanks for sharing!

With thread-safe you basically mean avoiding data race between threads, right? Or something else?

Have you ever had the need of using Atomic operations to avoid data race conditions? In the [Julia documentation](https://docs.julialang.org/en/v1/manual/multi-threading/#Atomic-Operations) atomic operations can become handy in these cases

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 21, 2024, 5:44pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/4 "2024-07-21T17:44:12Z")

</div>

> [@Gast](#):
>
> Have you ever had the need of using Atomic operations to avoid data race conditions? In the [Julia documentation](https://docs.julialang.org/en/v1/manual/multi-threading/#Atomic-Operations) atomic operations can become handy in these cases

Atomic operations are a very **low-level** thing, basically reserved for concurrency experts. In the majority of cases one should instead use higher level synchronization concepts.

---

<div class="post-metadata">

**Author:** ![Gast](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gast/32/206425_2.png) [@Gast](https://discourse.julialang.org/u/Gast)\
**Post date:** [July 21, 2024, 6:14pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/5 "2024-07-21T18:14:47Z")

</div>

Got it  
And what would be examples of higher level synchronization concepts?

---

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [July 21, 2024, 6:41pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/6 "2024-07-21T18:41:52Z")

</div>

> With thread-safe you basically mean avoiding data race between threads, right? Or something else?

I mean a slightly more general definition of thread-safety. If you try to write to a Dict from multiple threads, you won’t get races, you’ll get a segfault. From Wikipedia: " a function is **thread-safe** when it can be invoked or accessed concurrently by multiple threads without causing unexpected behavior, [race conditions](https://en.wikipedia.org/wiki/Race_condition), or data corruption."

> Have you ever had the need of using Atomic operations to avoid data race conditions?

I haven’t used Atomic – Channels are a higher-level concept, that along with Tasks are (theoretically) general enough express all concurrent computations.

Atomics seem more like they’re oriented towards language developers, or developers of particularly high performance libraries – they only work on primitive times, only support a limited range of operations, etc.

---

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [July 21, 2024, 6:59pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/7 "2024-07-21T18:59:47Z")

</div>

> And what would be examples of higher level synchronization concepts?

`Channel`, `Threads.@spawn`, and `@sync` are the main ones built in to Julia.

For higher-level options you can look at OhMyThreads.jl or ThreadsX.jl, both of which are under the JuliaFolds2 organization: [JuliaFolds2 · GitHub](https://github.com/JuliaFolds2)

---

<div class="post-metadata">

**Author:** ![sgaure](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sgaure/32/14779_2.png) [@sgaure](https://discourse.julialang.org/u/sgaure)\
**Post date:** [July 21, 2024, 9:05pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/8 "2024-07-21T21:05:19Z")

</div>

While `Channel`s are great for serializing access to shared resources like a common `Dict`, you could add a chapter about locks. It is somewhere between atomics and channels, and fairly easy to understand. What to use depends on how much overhead one can afford. Atomics isn’t very complicated either, except the fine points of memory ordering semantics.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 21, 2024, 9:22pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/9 "2024-07-21T21:22:05Z")

</div>

> [@Gast](#):
>
> examples

Atomics are at the lowest level. Above them are locks and semaphores, above those are condition variables (currently not available in Julia, AFAIK) and `Channel`.

See the Concurrency part of [Operating Systems: Three Easy Pieces](https://pages.cs.wisc.edu/~remzi/OSTEP/) for a general traditional overview. For semaphores in particular, see [The Little Book of Semaphores](https://greenteapress.com/wp/semaphores/) by AB Downey.

There was also this fun game, hope the Web site is still functional:

> **[The Deadlock Empire](https://deadlockempire.github.io)**
>
> Slay dragons, learn
> concurrency! Play the cunning Scheduler, exploit flawed
> programs and defeat the armies of the Parallel Wizard.

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [July 21, 2024, 9:28pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/10 "2024-07-21T21:28:08Z")

</div>

Your blog post says `@async` is deprecated? Julia’s official manual on [Network and Streams](https://docs.julialang.org/en/v1/manual/networking-and-streams/) uses `@async` in some examples. Should the examples be updated with `@spawn` as a drop-in replacement?

---

<div class="post-metadata">

**Author:** ![matthias314](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@matthias314](https://discourse.julialang.org/u/matthias314)\
**Post date:** [July 21, 2024, 10:25pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/11 "2024-07-21T22:25:57Z")

</div>

> [@greatpet](#):
>
> Julia’s official manual on [Network and Streams](https://docs.julialang.org/en/v1/manual/networking-and-streams/) uses `@async` in some examples.

In fact, `@async` appears 45 times in the Julia 1.10.4 manual and 48 times in the one for 1.11.0-rc1. Quite a bit for a deprecated macro!

---

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [July 21, 2024, 11:46pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/12 "2024-07-21T23:46:36Z")

</div>

> While `Channel` s are great for serializing access to shared resources like a common `Dict` , you could add a chapter about locks.

A post about this would be interesting for sure, but I’ve had a lot of trouble trying to write correct code with locks. E.g. we left code like this in our codebase for months, which was causing deadlocks in notebooks about once a week:

```julia
function setindex!(tsd::ThreadSafeDict, k, v)
    lock(tsd.slock)
    h = setindex!(tsd.d, k, v)
    unlock(tsd.slock)
    return h
end

```

In contrast when I wrote a Channel version of a thread-safe Dict it just worked.

> Your blog post says `@async` is deprecated? Julia’s official manual on [Network and Streams](https://docs.julialang.org/en/v1/manual/networking-and-streams/) uses `@async` in some examples. Should the examples be updated with `@spawn` as a drop-in replacement?

Probably? When I asked about it, I was told [“for new code there is no reason to use `@async`”](https://discourse.julialang.org/t/examples-of-when-to-use-async-over-spawn/114631).

---

<div class="post-metadata">

**Author:** ![era127](https://avatars.discourse-cdn.com/v4/letter/e/eb8c5e/32.png) [@era127](https://discourse.julialang.org/u/era127)\
**Post date:** [July 22, 2024, 2:18am UTC](https://discourse.julialang.org/t/using-tasks-101/114878/13 "2024-07-22T02:18:17Z")

</div>

It might be worth mentioning ThreadSafeDicts.jl as well.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 22, 2024, 4:21am UTC](https://discourse.julialang.org/t/using-tasks-101/114878/14 "2024-07-22T04:21:49Z")

</div>

> [@nsajko](#):
>
> For semaphores in particular, see [The Little Book of Semaphores](https://greenteapress.com/wp/semaphores/) by AB Downey.

Fwiw: [GitHub - carstenbauer/LittleBookOfSemaphores.jl: Julia code snippets inspired by the Little Book Of Semaphores · GitHub](https://github.com/carstenbauer/LittleBookOfSemaphores.jl)

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 29, 2024, 7:17am UTC](https://discourse.julialang.org/t/using-tasks-101/114878/15 "2024-07-29T07:17:17Z")

</div>

> [@greatpet](#):
>
> Your blog post says `@async` is deprecated? Julia’s official manual on [Network and Streams](https://docs.julialang.org/en/v1/manual/networking-and-streams/) uses `@async` in some examples.

> <https://github.com/JuliaLang/julia/issues/50356>
>
> The \`@async\` macro documentation includes the following \[warning\](https://github….com/JuliaLang/julia/blame/f6f35533f237d55e881276428bef2f091f9cae5b/base/task.jl#L495):
> 
> \> It is strongly encouraged to favor \`Threads.@spawn\` over \`@async\` always even when no parallelism is required especially in publicly distributed libraries. This is because a use of \`@async\` disables the migration of the parent task across worker threads in the current implementation of Julia. Thus, seemingly innocent use of \`@async\` in a library function can have a large impact on the performance of very different parts of user applications.
> 
> However, the asynchronous programming piece of the manual presents \`@async\` as a legitimate way to do things - and \`@async\` is even used in the how-to related to Channels. Take a look \[here\](https://github.com/JuliaLang/julia/blame/f6f35533f237d55e881276428bef2f091f9cae5b/doc/src/manual/asynchronous-programming.md#L67C36-L67C36) and in general to the asynchronous programming section of the manual.
> 
> From the timestamps, it seems that the asynchronous programming section of the manual might be outdated - and is basically suggesting practices that are labeled as to be avoided in other parts of the documentation.
> 
> I also opened a topic on \[discourse\](https://discourse.julialang.org/t/inconsistency-between-async-warning-and-its-usage-in-documentation/100978?u=algunion), but it didn't grab much attention.
> 
> Is the \`@async/@swap\` warning still standing and the manual is to be updated? Or the other way around?

> <https://github.com/JuliaLang/julia/issues/52109>
>
> \<!--
> If you have a question please search or post to our Discourse site: https:…//discourse.julialang.org.
> We use the GitHub issue tracker for bug reports and feature requests only.
> 
> If you're submitting a bug report, be sure to include as much relevant information as
> possible, including:
> 
> 1. The output of \`versioninfo()\`
> 2. How you installed Julia
> 3. A minimal working example (MWE), also known as a minimum reproducible example
> 
> If you're experiencing a problem with a particular package, open an issue on that
> package's repository instead.
> 
> Thanks for contributing to the Julia project!
> \--\>
> Julia does need the ability to pin a task to a thread (in our current terminology: make it sticky). This currently happens implicitly for tasks created with \`Base.@async\` to \[support its semantics\](https://github.com/JuliaLang/julia/issues/41324). To support those semantics, the parent task must also be \[made sticky\](https://github.com/JuliaLang/julia/blob/f31cd8ad2fecefa2717aa0f5462876c985acbdea/base/task.jl#L794). More precisely, if a newly created task \`S\` is sticky (which they are when created with \`Task()\` or \`@task\`), then whichever task calls \`schedule(S)\` will also be made sticky. A task that calls the \`AsyncCondition(callback)\` constructor (which spawns an \`@async\` task) will become sticky, as will a task that calls \`Timer(callback)\`.
> 
> This sort of stickiness-infection ignores threadpool boundaries -- a task created on the default threadpool can end up being stuck to an interactive thread.
> 
> We can fix some of these (as we have \[previously\](https://github.com/JuliaLang/julia/pull/49601)), but we cannot fix them all, due to \`@async\` semantics.
> 
> Furthermore, such stickiness-infected tasks are \[never\](https://github.com/JuliaLang/julia/blob/f31cd8ad2fecefa2717aa0f5462876c985acbdea/base/task.jl#L785) \[unstuck\](https://github.com/JuliaLang/julia/blob/f31cd8ad2fecefa2717aa0f5462876c985acbdea/base/task.jl#L331). There is \[this PR\](https://github.com/JuliaLang/julia/pull/41393) which attempts to unstick such tasks, but adds too much overhead.
> 
> We should introduce an explicit task pin/unpin interface (see https://github.com/JuliaLang/julia/issues/52108) and deprecate \`Base.@async\` so that all this implicit stickiness and associated cruft can be removed from the runtime.

---

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [July 30, 2024, 7:49pm UTC](https://discourse.julialang.org/t/using-tasks-101/114878/16 "2024-07-30T19:49:31Z")

</div>

Ok, I’ve attempted a PR to update the documentation, let’s see how this goes: [Replace `@async` mentions in manual with `Threads.@spawn` by Satvik · Pull Request #55315 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/55315)