Difference between nprocs and nthreads

Specific Domains Julia at Scale
1

Hello,

I’m trying to get my head around distributed programming to execute a script on a cluster.
My code works well on my workstation with one CPU. It internally uses @threads to parallelize some stuff within the main task, using the threads of my one CPU. So I start as follows: $ julia -t 12.

On the cluster, I would like to launch several of this task (each will use @threads occasionally).
Now I’m reading the documentation of distributed programming and I read

Starting with julia -p n provides n worker processes on the local machine. Generally it makes sense for n to equal the number of CPU threads (logical cores) on the machine. Note that the -p argument implicitly loads module Distributed.

Say I have 4 CPUs on my cluster node. Should I start with

  1. $ julia -p 4 -t 12 (assuming 12 cores per cpu) or
  2. $ julia -p 48 ? The latter takes the above quote literally.

I tried 2) on my local machine and I see that nthreads() == 1. 1) gives nprocs() == 4 && nthreads == 12.

Am I correct to think that with 1) I can start 4 parallel “main” tasks" that will each use 12 threads for multithreading or will it be 3 because the 12 threads are divided between the workers ? Meaning I should use $ julia -p 4 -t 48 instead.

and

Am I correct to think that with 2) I will launch 48 “main” tasks that will not be able to multithread my @threads calls because nthreads() == 1 then?

1 Like
2

Do @everywhere @show Threads.nthreads().

This runs the code on every process, so you’ll see how many threads they each have.
They do not share with one another.

3

Thank you :slight_smile: