# Poll: What do you use: Distributed or MPI?

**URL:** <https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741>\
**Category:** General Usage\
**Created:** [August 20, 2025, 3:58pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741 "2025-08-20T15:58:56Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![linwaytin](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@linwaytin](https://discourse.julialang.org/u/linwaytin)\
**Post date:** [August 20, 2025, 3:58pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/1 "2025-08-20T15:58:56Z")

</div>

Julia’s default multi-processing system, `Distributed`, is based on a different paradigm than the de facto standard `MPI` in HPC.

I’m really wondering the numbers of people using the two different types of distributed computation. If you use both of them frequently, please select both.

I also appreciate if people can share their experience with these 2 different architectures.

_Poll ([view on site](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/1))_

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 21, 2025, 1:17am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/2 "2025-08-21T01:17:10Z")

</div>

Distributed has a big disadvantage: slow communication between processes. MPI implementations typically optimize that a lot.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 21, 2025, 6:47am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/3 "2025-08-21T06:47:07Z")

</div>

> [@PetrKryslUCSD](#):
>
> Distributed has a big disadvantage: slow communication between processes.

Does DistributedNext improve this?

---

<div class="post-metadata">

**Author:** ![linwaytin](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@linwaytin](https://discourse.julialang.org/u/linwaytin)\
**Post date:** [August 21, 2025, 6:40pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/4 "2025-08-21T18:40:52Z")

</div>

> [@PetrKryslUCSD](#):
>
> Distributed has a big disadvantage: slow communication between processes

I am curious how slow the communication is. Do you have some numbers?

---

<div class="post-metadata">

**Author:** ![xzackli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xzackli/32/38301_2.png) [@xzackli](https://discourse.julialang.org/u/xzackli)\
**Post date:** [August 22, 2025, 12:13am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/5 "2025-08-22T00:13:24Z")

</div>

I use MPI because

1. CUDA works with it (but I have to be careful with compiling PTX kernels ahead of time)
2. HDF5 works with it, which lets me do very large-scale reads and writes very conveniently.
3. PencilFFTs.jl and PencilArrays.jl are really excellent packages for MPI

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 22, 2025, 3:04am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/6 "2025-08-22T03:04:37Z")

</div>

I’m afraid I don’t have any up-to-date numbers to support my claim. I did some testing two or three years ago, and it was no contest: MPI won handily. However, YMMV. It depends on the granularity of the computation. Little communication and lots of computation may well favor Distributed.

I am not familiar with DistributedNext. I’ll be curious to see what’s there.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 22, 2025, 4:00pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/7 "2025-08-22T16:00:58Z")

</div>

I use both when developing Dagger.jl, because users might have a reason for using one or the other. Some users want the dynamism and flexibility of Distributed, while others want the raw performance and scalability of MPI. Some users want both, and use MPIClusterManager, and are thus using both at the same time!

For reference, in Dagger, we provide high-level abstractions (like our `DArray`, and our Datadeps parallel algorithm framework) that don’t specify one or the other, but allow users to tell Dagger which to use, while providing equivalent semantics regardless of which option is chosen. This means the problem “Does library XYZ support Distributed or MPI?” doesn’t really matter when the library supports Dagger; the difference is handled behind the scenes, and no code has to be rewritten.

(Also, we allow selecting between Distributed and DistributedNext, so regardless of which Distributed-compatible library users choose, everything “just works”)

---

<div class="post-metadata">

**Author:** ![linwaytin](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@linwaytin](https://discourse.julialang.org/u/linwaytin)\
**Post date:** [August 22, 2025, 10:39pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/8 "2025-08-22T22:39:29Z")

</div>

Cool. Is `Dagger` easy to use than `MPI`?

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 22, 2025, 11:01pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/9 "2025-08-22T23:01:44Z")

</div>

I’d definitely say so! MPI makes you deal with details like parallel synchronization and coordination, data transfer/serialization, managing concurrency, etc. - all complicated things that even HPC experts struggle with.

Dagger (Datadeps specifically) takes the path of letting you express your program as a serial sequence of operations on a partitioned array, which Dagger then parallelizes for you based on little data dependency annotations. Dagger then handles those previously-mentioned complexities for you, efficiently utilizing MPI for data transfers and automatically scheduling your program across ranks and managing concurrency and program ordering, so that you still get the performance you expect from a well-written MPI program, without the mental overhead of having to do the MPI calls (and everything else) yourself.

---

<div class="post-metadata">

**Author:** ![linwaytin](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@linwaytin](https://discourse.julialang.org/u/linwaytin)\
**Post date:** [August 24, 2025, 1:18am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/10 "2025-08-24T01:18:38Z")

</div>

That sounds amazing! I feel more description about `Dagger` should be added to [Multi-processing and Distributed Computing · The Julia Language](https://docs.julialang.org/en/v1/manual/distributed-computing)

---

<div class="post-metadata">

**Author:** ![souma](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/souma/32/212836_2.png) [@souma](https://discourse.julialang.org/u/souma)\
**Post date:** [August 24, 2025, 10:15pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/11 "2025-08-24T22:15:29Z")

</div>

I used `Distributed` largely because most of my work is embarrassingly parallel (thus the advantage of `MPI` communication being moot) and my collaborators are not great with programming, and `Distributed` is basically a better version of R’s `foreach` and `doParallel` packages.

Plus writing code in `MPI` hurty my brainy.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [August 25, 2025, 5:44am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/12 "2025-08-25T05:44:38Z")

</div>

I’ve never done raw MPI in Julia, but I use `MPIManager` a fair bit to set up the workers via MPI then use `Distributed`-type calls (plus this gets you faster inter-communication and benefits of CUDA-aware MPI for GPU work if the cluster is set up right)

---

<div class="post-metadata">

**Author:** ![fverdugo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fverdugo/32/9446_2.png) [@fverdugo](https://discourse.julialang.org/u/fverdugo)\
**Post date:** [August 25, 2025, 8:01am UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/13 "2025-08-25T08:01:24Z")

</div>

Performance aside, Distributed and MPI follow quite different paradigms. Some applications might benefit more from one than the other.

For instance, using Distributed is straightforward to implement a [replicated workers model](https://en.wikipedia.org/wiki/Thread_pool), while with MPI is easier to use many common communication patterns in scientific computing as they are already implemented for you (and with good hardware support).

I would say that virtually any distributed algorithm can be implemented using any of both approaches. But one can be more practical than the other depending on your particular use case.

---

<div class="post-metadata">

**Author:** ![linwaytin](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@linwaytin](https://discourse.julialang.org/u/linwaytin)\
**Post date:** [August 25, 2025, 5:37pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/14 "2025-08-25T17:37:03Z")

</div>

> [@fverdugo](#):
>
> Performance aside, Distributed and MPI follow quite different paradigms. Some applications might benefit more from one than the other.

That is a great point!

---

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 27, 2025, 10:57pm UTC](https://discourse.julialang.org/t/poll-what-do-you-use-distributed-or-mpi/131741/15 "2025-08-27T22:57:51Z")

</div>

> [@linwaytin](#):
>
> I also appreciate if people can share their experience with these 2 different architectures.

There is a paper from 2021 by Amal Rizvi and Kyle C. Hale titled “[A Look at Communication-Intensive Performance in Julia](https://arxiv.org/abs/2109.14072)”. It does not fully compare Distributed and MPI, but you might find it interesting. There is also a discussion on a similar topic from 2021 [here](https://discourse.julialang.org/t/distributed-jl-vs-mpi-jl/71315).