I want to add an update and solution to this thread.
The answer by @ElOceanografo is great, specially if you are interested in fine control with the cluster resource manager. I would say that the most clean solution nowadays is through the ClusterManagers.jl package.
Basically, the package defines various functions addprocs_slurm()
, addprocs_pbs()
, etc. which can be used in place of the Julia builtin addprocs()
. When you start Julia, instead of asking for processes with addprocs()
, you simply do:
using ClusterManagers
addprocs_pbs(100) # request 100 processes in the cluster using PBS
# REST OF SCRIPT GOES HERE
This will take care of submitting the job with the appropriate resource manager without having to write the job script manually.