What is julia doing with your threads?

General Usage
20

I can replicate this as well on my laptop with Julia 1.10.0 and a AMD Ryzen 7 4800H

  • -t 5 gives extrema(ts) = (1.126322503, 2.474583519) → Sometimes slow and no interactive threads
  • -t 5,1 gives extrema(ts) = (1.130685362, 1.339060755) → never slow and 1 interactive thread
  • -t auto (equivalent to -t 16,0 for me) gives:
    extrema(ts) = (1.148305388, 2.280251291) → sometimes slow and no interactive threads

I wonder if this is loosely related to Bug in sleep() function - main thread work affecting sleep duration on running tasks.
Here is a wild guess (as I really don’t anything about the implementation of Tasks and scheduling): Julia needs to schedule the task in some thread. If there are no interactive threads, Julia’s “main thread” is in the same threadpool that works on the tasks. So maybe sometimes a Task gets scheduled on the main thread and starts running before all Tasks where scheduled and so some Tasks are scheduled late. This would not happen if there is at least a single interactive thread because the main thread is always in the interactive pool[1] and the tasks are scheduled to run in the :default pool.
I modified the example above to schedule the tasks on the :interactive pool instead and then the slowdown again occurs. For julia -t 5,5 (so 5 interactive threads and 5 normal ones) I get again extrema(ts) = (1.117211121, 2.510383934).

EDIT: Found another thread that notices (likely) this problem: With julia-1.9, should the main task block :interactive tasks? - #2 by samtkaplan

  1. This can be verified by starting a session with or without interactive threads and just running Threads.threadpool() to see the current threadpool. It is :interactive if there are interactive threads and :default otherwise. ↩︎

2 Likes