# Resources to Understand Distributed.jl

**URL:** <https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395>\
**Category:** New to Julia\
**Created:** [December 11, 2023, 12:14am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395 "2023-12-11T00:14:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bdas123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdas123/32/32142_2.png) [@bdas123](https://discourse.julialang.org/u/bdas123)\
**Post date:** [December 11, 2023, 12:14am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/1 "2023-12-11T00:14:16Z")

</div>

Besides the Distributed.jl documentation, are there any other good resources to understand the theory/application of the library? I’d figure to ask here before I do a deep dive on the topic.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [December 11, 2023, 12:19am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/2 "2023-12-11T00:19:34Z")

</div>

I think @jpsamaroo recommends Dagger.jl instead of Distributed.jl.

---

<div class="post-metadata">

**Author:** ![bdas123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdas123/32/32142_2.png) [@bdas123](https://discourse.julialang.org/u/bdas123)\
**Post date:** [December 11, 2023, 12:21am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/3 "2023-12-11T00:21:48Z")

</div>

Oh interesting. What’s the reason for that?

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [December 11, 2023, 1:17am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/4 "2023-12-11T01:17:07Z")

</div>

Distributed.jl doesn’t provide a whole lot of (composable) tools for doing distributed parallelism, in my opinion. `pmap` is pretty limited to doing just `map` operations, and `@distributed` performs reductions, but that’s really it. If you wanted to do parallel computing on any “real” data structure, like an array, table, or graph, you need to reach for something more capable (like Dagger.jl, which provides distributed implementations for all 3 respective data structures).

Additionally, Distributed.jl fails to provide any useful abstractions for orchestrating data movement or remote task execution, which is important to be able to do any kind of more complicated distributed parallelism. It would be like trying to do multithreaded programming without tasks or built-in atomics - you could do it, but you’d have to do a lot of work yourself to fill in what’s missing. Again, Dagger provides abstractions for data management and provides tasks which can seamlessly execute across a cluster of Julia processes.

Still, it’s not like Distributed.jl is inherently _bad_ - Dagger.jl uses it for all of its distributed operations, to move data and execute functions remotely. But it’s just not a great _user-facing_ interface - it really should be hidden behind a more user-friendly library, like Dagger.jl, where it can be a platform to build upon.

---

<div class="post-metadata">

**Author:** ![bdas123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdas123/32/32142_2.png) [@bdas123](https://discourse.julialang.org/u/bdas123)\
**Post date:** [December 11, 2023, 1:23am UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/5 "2023-12-11T01:23:49Z")

</div>

Awesome to know!

I was having problems implementing Distributed.jl yesterday, so I’ll look into Dagger.jl

---

<div class="post-metadata">

**Author:** ![bdas123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdas123/32/32142_2.png) [@bdas123](https://discourse.julialang.org/u/bdas123)\
**Post date:** [December 11, 2023, 1:50pm UTC](https://discourse.julialang.org/t/resources-to-understand-distributed-jl/107395/6 "2023-12-11T13:50:05Z")

</div>

@jpsamaroo, I would greatly appreciate if you could take a look at my first question on Dagger.jl: [Not seeing speed-up with Dagger.jl. Am I doing something wrong?](https://discourse.julialang.org/t/not-seeing-speed-up-with-dagger-jl-am-i-doing-something-wrong/107440)