Julia distributed and multithreaded

I think in your case, as the workloads are completely separate (one after the other), there’s no reason you can’t have 8 threads and 8 processes at the same time, especially as it’s unlikely that your external library will use multithreading and multiprocessing at the same time.

Also, you don’t have to start the processes from the command line:

julia -t 8

Then:

using Distributed
addprocs(8,exeflags=["--threads=1"])

Which makes the additional processes have only a single thread. You will likely only ever be using multiprocessing or multithreading, and not both and so can benefit from parallelism in your code and the library code.

2 Likes