How to start a julia process on a remote machine with multiple threads?

General Usage
1

When starting a Julia session with the following command julia --machine-file machine-file.txt -p 2, the processes created on the remote machine have only one thread, even with the environment variable JULIA_NUM_THREADS defined as 8 in the local machine and the machine.

2

Just a quick reply, there is the CMD option -t for the number of threads. Multi-Threading · The Julia Language Notice the difference between threads and processes…

E.g. does julia -t 8 ... work? (Not sure why the JULIA_NUM_THREADS doesn’t work. Maybe it depends on the way you connect to the remove machine.)

1 Like
3

Starting with julia -tn -pm --machine-file=etc. does indeed get you n threads on each remote machine.

MORE DETAILS:

Just for completeness…In simple cases your -tn flag is propagated to the remote workers. But if you want to use different numbers of threads on different workers, you can use addprocs() (Distributed Computing · The Julia Language):

julia> addprocs([("sp4", 2)]; exeflags="-t 3");
                       # ↑ 2 worker processes on sp4, please
julia> fetch(@spawnat 3 readchomp(`hostname`))
"sp4"

julia> fetch(@spawnat 3 Threads.nthreads())
3

julia> Threads.nthreads()
1

Here “sp4” is the hostname of another computer in my apartment that has Julia installed.

2 Likes
4

It works. Thank you guys.