Number of threads assigned via SLURM

Specific Domains
,
1

I am trying to run some multi-threaded code on a cluster using SLURM.

#SBATCH --cpus-per-task=4
JPATH=$HOME/.juliaup/bin
export JULIA_DEPOT_PATH="/my/storage/dir/.julia:"
export JULIA_NUM_THREADS=auto
$JPATH/julia main_0_test.jl

Using ThreadPinning.jl, I pin the threads according to affinity mask and then check threadinfo with slurm=true.

Hostname: 	r184.ib.bridges2.psc.edu
CPU(s): 	2 x AMD EPYC 7742 64-Core Processor
CPU target: 	znver2
Cores: 		128 (128 CPU-threads)
NUMA domains: 	2 (64 cores each)

SLURM: 2 assigned CPU-threads. Will only show those below.

Julia threads: 	4

CPU socket 1
  4,5


# = Julia thread, # = >1 Julia thread

(Mapping: 1 => 124, 2 => 125, 3 => 126, 4 => 127,)
Threads.nthreads() = 4

I am confused as to why it says slurm only assigned 2 cpu threads. Number of Julia threads are correct, and the mapping also shows where they have been assigned. The computations also run faster as I increase the number of cpus requested. I am wondering what does the “2 assigned CPU-threads” mean and if it will lead to collisions when running multiple jobs. Would it be better to ask these questions to the team running the clusters instead?

2

My first thought would be that the chip has hyperthreads (or whatever the AMD equivalent is), so Julia pins its 4 threads to 4 logical cores over 2 physical cores and SLURM only allocates or sees the 2 physical. But that’s a complete guess as I’m mostly unfamiliar with SLURM.

Is the program using SlurmClusterManager.jl?