For some analysis where each thread is memory intensive on separate memory, using all logical cores might be suboptimal because then pairs of threads (hyperthreading) on a single core might be thrashing the caches. It seems likely for this case that threading based on the physical cores and caches would be more optimal. Is there any way in Julia to deal with this?
It might be as simple as using the same number of threads that there are physical cores which I could get from Hwloc.jl. However, there is no way with @threads to specify how many threads to use. Setting the number of threads on the command line is not ideal because I might want a different number in the same session.
ThreadPools.jl has a mechanism to specify the number of threads if you explicitly create a thread pool. Also, you could schedule on a specific thread id. However, does a thread id correspond to a logical core, or is that just used for Julia’s housekeeping and the CPU is free to schedule any thread on any logical core? Similarly, does Julia currently support anything regarding thread/core affinity?
(To be a bit more precise, it is worth mentioning that the number of threads can actually change at runtime since Julia 1.9 because we added a thread adoption feature. However, at least for now, this is intended to be used to adopt “foreign” threads, e.g. when running Julia in conjunction with a C code. You could perhaps exploit this feature to change the number of threads dynamically already right now. But there definitely is no high-level API for this yet.) I’d like to see proper support for this in the future.
As @dlakelan mentioned, ThreadPinning.jl is your friend. I recently gave a lightning-talk about it at JuliaCon 2023, which is available on Youtube (not yet as a dedicated video but that will come soon as well).
That is correct, only Linux is supported right now. Note that macOS will likely never be supported because it just fundamentally doesn’t allow users to pin threads. There is nothing I can do about it. For Windows, the situation is different and I even started a PR for adding Windows support at some point. It stalled because I don’t have good access to a Windows machine and, for the same reason, don’t care enough about it (I mostly work on HPC clusters these days which all run Linux). I would appreciate it though if someone would revive the PR.
(FWIW, at JuliaCon, @miguelraz has indicated that he might want to help out here.)