How does one pass topology to a cluster manager? Julia docs mention the setup of topology in the case when
addprocs dispatches on
machines, but not on a cluster manager.
I am trying to do the following thing.
- I would like to create a job (with multiple processes) for each of the computing nodes I require via
- For each of the jobs, I would like to have a master with a bunch of workers
- The parent process that ran
addprocsshould connect only to the master process in each of the nodes.
In essence, I want to create a “chain of command”: parent process would give instructions to the master processes on each of the nodes; these masters would give instructions to their respective workers.
The reason why I am trying to do that is the following: I need to share the structures between the workers from the same computing node.