Multiple Worker Processes with Different Numbers of Threads

General Usage
1

Hi all,

I’m trying to write a program which needs to solve three large problems and about 50 small problems in parallel. I’m planning to create workers for each process, but need 8 threads for the large problems and one for each of the small problems. Any suggestions as to how would I do something like this in julia?

Thanks for your help!

2

For what you want , Distributed.jl should have you covered I think. addprocs supports the keyword exeflags which allows you to specify the amount of threads the worker process should be started with.

Is there a reason you require separate processes for though? Should these workers be distributed to different machines or do you need to use external libraries that can’t be parallelized otherwise? If not, then I’d recommend just using a single Julia process with however many threads you can afford and letting Julia’s scheduler handle the rest :slight_smile:

3

Thank you! This is very helpful. As you can tell I’m a novice at distributed programs. Each process will be solving an LP using Gurobi, which is automatically multithreaded, which is why I requested the layout I did. However if you have suggestions for improved layouts, I’d love to hear it!

Thanks again!