# How to get started with distributed memory parallel programming?
**URL:** https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617
**Category:** New to Julia
**Created:** [June 9, 2021, 9:46am UTC](https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617 "2021-06-09T09:46:42Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@greatpet](https://discourse.julialang.org/u/greatpet)
#### Post date: [June 9, 2021, 9:46am UTC](https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617/1 "2021-06-09T09:46:42Z")
I’ve used OpenMP in C/C++ and multi-threading in Julia, but have never learned MPI or other distributed frameworks (excepting submitting arrays of jobs to clusters for trivially parallelizable problems). What is the “Julian” way to do distributed memory parallel programming? What are the recommended packages?
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [June 9, 2021, 9:56am UTC](https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617/2 "2021-06-09T09:56:06Z")
Short answer: Julia has built-in distributed computing functionality (see [Multi-processing and Distributed Computing · The Julia Language](https://docs.julialang.org/en/v1/manual/distributed-computing/) and maybe [Jupyter Notebook Viewer](https://nbviewer.jupyter.org/github/crstnbr/JuliaNRW21/blob/master/Day3/1_parallel_computing.ipynb)). Otherwise, there is [GitHub - JuliaParallel/MPI.jl: MPI wrappers for Julia](https://github.com/JuliaParallel/MPI.jl).
My opinion: Julias built-in features are great for small scale parallelisation of certain tasks. If you want to do actual HPC across hundreds of nodes of a supercomputer I’d go with MPI.jl because it utilises infiniband and (please someone correct me if I’m wrong) no one has really pushed Julias built-in distributed computing facilities to the limit and managed or proven to be competitive. (If anyone than perhaps the [https://clima.caltech.edu](https://clima.caltech.edu) people?)
---
### Author:  [@greatpet](https://discourse.julialang.org/u/greatpet)
#### Post date: [June 9, 2021, 10:26am UTC](https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617/3 "2021-06-09T10:26:25Z")
Skimming through your first link, I noticed that Julia launches remote worker via password-less SSH. How do I let Julia know which cluster nodes are available for use, given that only a subset of nodes is assigned to me by the cluster queuing system when I request a multi-node run? I suppose ClusterManager.jl can launch workers one by one by submitting separate jobs to the queue, but this is not very feasible given the unpredictable wait time in the queue, and I would prefer to use a bunch of machines assigned to me in a single multi-node request. Or do I inevitably end up with MPI.jl, since the `mpirun` command will know which nodes to use?
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [June 9, 2021, 10:39am UTC](https://discourse.julialang.org/t/how-to-get-started-with-distributed-memory-parallel-programming/62617/4 "2021-06-09T10:39:01Z")
> [@greatpet](#):
>
> I suppose ClusterManager.jl can launch workers one by one by submitting separate jobs to the queue, but this is not very feasible given the unpredictable wait time in the queue, and I would prefer to use a bunch of machines assigned to me in a single multi-node request.
No, say you use the `SlurmManager`, it uses `srun` to start workers within your SLURM allocation (came up [here](https://discourse.julialang.org/t/issues-with-machinefile-and-slurm/7882/2) as well). It doesn’t submit a new job to the queue. Note that SLURM also stores the allocations details (which nodes, how many cpus etc.) in environment variables (e.g. `SLURM_JOB_NODELIST`) that you can query in your job submission script and/or Julia itself. See for example the section “OUTPUT ENVIRONMENT VARIABLES” [here](https://slurm.schedmd.com/sbatch.html) or the table [here](https://hpcc.umd.edu/hpcc/help/slurmenv.html).