# Controlling thread affinity for distributed multithreading

**URL:** https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869
**Category:** Julia at Scale
**Created:** [January 24, 2021, 6:11pm UTC](https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869 "2021-01-24T18:11:30Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [January 24, 2021, 6:11pm UTC](https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869/1 "2021-01-24T18:11:30Z")

</div>

I have a question regarding the setup of hybrid parallelism which combines multithreading and distributed computations.

Let’s say I want to run some computation on a machine with Non-Uniform Memory Access. In this case, it is a good idea to have a julia process per NUMA node, each of the processes employing multiple threads which run on the same NUMA node.

I know that `addprocs` allows one to specify the cpu affinity of the additional julia processes: these way I can pin the processes to different NUMA nodes. Besides that, one can pass to `addprocs` special flag to use multithreading for these processes. **My question is: how does one specifies the affinity of these threads? (So that they are pinned to the same NUMA node as the process they belong to.)**

---

<div class="post-metadata">

### Author: ![dkiese](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dkiese](https://discourse.julialang.org/u/dkiese)
#### Post date: [May 12, 2021, 9:07am UTC](https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869/2 "2021-05-12T09:07:09Z")

</div>

I have been using numactl to set the thread affinity like

```julia
numactl --physcpubind=0-$(cpus_per_task - 1) -- $(exe) -O3 $(input) -E 'run(`numactl -s`)'

```

where `exe` was the path to my Julia executable and `input` the script to be executed. Last time I checked on our local cluster, the processor affinity carried over to the threads controlled by different Julia processes, pinning one thread to one core on every compute node. It’s been a while though …

---

<div class="post-metadata">

### Author: ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [May 12, 2021, 12:00pm UTC](https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869/3 "2021-05-12T12:00:52Z")

</div>

See also [Thread affinitization: pinning Julia threads to cores](https://discourse.julialang.org/t/thread-affinitization-pinning-julia-threads-to-cores/58069)

---

<div class="post-metadata">

### Author: ![dkiese](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dkiese](https://discourse.julialang.org/u/dkiese)
#### Post date: [May 12, 2021, 12:44pm UTC](https://discourse.julialang.org/t/controlling-thread-affinity-for-distributed-multithreading/53869/4 "2021-05-12T12:44:49Z")

</div>

Wasn’t aware that `JULIA_EXCLUSIVE` exists, thanks for linking the other issue.
