# IO Parallelism Strategies

**URL:** <https://discourse.julialang.org/t/io-parallelism-strategies/120970>\
**Category:** General Usage\
**Tags:** multithreading, io\
**Created:** [October 6, 2024, 11:09am UTC](https://discourse.julialang.org/t/io-parallelism-strategies/120970 "2024-10-06T11:09:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [October 6, 2024, 11:09am UTC](https://discourse.julialang.org/t/io-parallelism-strategies/120970/1 "2024-10-06T11:09:37Z")

</div>

I have a use case with many small IOs (get/put S3 objects \<300mb), a heartbeat (keep SQS message from timing out), and a heavy compute task which consumes a constant stream of inputs. The machine I’m using has 2 cores.

Recognizing that the real answer is “profile it”, what is your intuition for the best parallelism strategy? There are so many knobs to turn, and so much randomness created by the networking that I don’t think approaching optimization with pure experimentation is going to be fruitful.

I am trying to achieve:

1. The compute task is never waiting for more inputs or for the results to be put back to S3
2. The heartbeat isn’t blocked for so long that it fails to update the timeout.

One thing that is clear is that the main loop should kick off tasks for IO and heartbeat so that that they do not block execution of the heavy compute. However, there are several consideration for this.

1. How do I make sure the heart beat isn’t blocked for a long time by a long list of IO tasks?
2. I’ve seen (maybe outdated) posts that libuv poses some fundamental challenges to concurrent IO because of its global lock. AWS.jl suggests switching to the Downloads.jl backend (instead of HTTP.jl) if you need to do concurrent requests. Unfortunately, [Curl.jl is segfaulting for me periodically](https://github.com/JuliaLang/Downloads.jl/issues/260), so I’m still using HTTP.jl until the patch mentioned in that issue makes it’s way to me. I’m not sure how this should affect my strategy (maybe `@async` is better than `@spawn` for this case? Would the advice change once I can move to Downloads.jl?)
3. I know that, under most circumstances, It is not advisable to have more Julia threads than logical cores. However, in this case where I have an ongoing heart beat that is very light weight but also needs to not be blocked, maybe I should run Julia with one `:interactive` and two `:default` threads and only schedule the heart beat on the interactive thread.

My current idea is to `@spawn` the heart beats to an `:interactive` thread, spawn an IO manager task which `@async` schedules all IO operations and leave the heavy compute on the main thread. But I’m open to the idea that I’m overthinking this and I should just `@spawn` everything naively.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [October 6, 2024, 5:36pm UTC](https://discourse.julialang.org/t/io-parallelism-strategies/120970/2 "2024-10-06T17:36:56Z")

</div>

> [@mrufsvold](#):
>
> My current idea is to `@spawn` the heart beats to an `:interactive` thread

We tried this for exactly this use-case, heartbeat bumping a SQS timeout, and I think it helped but did not totally solve task starvation issues. Maybe @omus or @dave.f.kleinschmidt remembers more; I think maybe we just bumped the timeout amount so there’s more leeway.

> [@mrufsvold](#):
>
> . [AWS.jl](https://juliahub.com/ui/Packages/General/AWS) suggests switching to the [Downloads.jl](https://juliahub.com/ui/Packages/General/Downloads) backend (instead of [HTTP.jl](https://juliahub.com/ui/Packages/General/HTTP)) if you need to do concurrent requests

I think this is stale, I just filed [rm stale notes about HTTP.jl concurrency issues by ericphanson · Pull Request #694 · JuliaCloud/AWS.jl · GitHub](https://github.com/JuliaCloud/AWS.jl/pull/694) to update those docs; that was only true pre-HTTP.jl 1.0 I believe.

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [October 6, 2024, 8:01pm UTC](https://discourse.julialang.org/t/io-parallelism-strategies/120970/3 "2024-10-06T20:01:14Z")

</div>

When you say you tried this, were you also using one more Julia threads than you had logical cores in your machine?

And, thanks for the update about HTTP.jl! That’s great news!

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [October 6, 2024, 9:14pm UTC](https://discourse.julialang.org/t/io-parallelism-strategies/120970/4 "2024-10-06T21:14:46Z")

</div>

> [@mrufsvold](#):
>
> When you say you tried this, were you also using one more Julia threads than you had logical cores in your machine?

No, using `n` regular threads + 1 interactive thread on a `n+1` core machine, so the total Julia threads matches the number of cores