# How well does Julia support distributed processing?

**URL:** <https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312>\
**Category:** New to Julia\
**Tags:** distributed\
**Created:** [September 17, 2021, 6:57am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312 "2021-09-17T06:57:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![stefkuypers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefkuypers/32/19185_2.png) [@stefkuypers](https://discourse.julialang.org/u/stefkuypers)\
**Post date:** [September 17, 2021, 6:57am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/1 "2021-09-17T06:57:24Z")

</div>

Hi,

We’re looking at using Julia for large scale simulations and are wondering how well Julia supports distributed processing. Where do we need to look for libraries supporting this? Are there any example projects we could look at?

Thanks in advance,  
Stef Kuypers

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 17, 2021, 7:16am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/2 "2021-09-17T07:16:03Z")

</div>

There’s the [Distributed](https://docs.julialang.org/en/v1/stdlib/Distributed/) stdlib, on top of which [ClusterManagers.jl](https://juliahub.com/ui/Packages/ClusterManagers/2gedf/0.4.2) is built. There’s also “regular” [MPI](https://juliahub.com/ui/Packages/MPI/nO0XF/0.19.0) if you want to do the process management manually (though I prefer ClusterManagers). [Transducers.jl](https://juliahub.com/ui/Packages/Transducers/cATK6/0.4.65) is also worth mentioning here.

More specific advise will depend on what kind of processing you’re doing, what your environment looks like etc. “Distributed Computing” is a kind of vague term, so do you have a certain application/usecase in mind?

---

<div class="post-metadata">

**Author:** ![stefkuypers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefkuypers/32/19185_2.png) [@stefkuypers](https://discourse.julialang.org/u/stefkuypers)\
**Post date:** [September 17, 2021, 8:04am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/3 "2021-09-17T08:04:25Z")

</div>

Thanks!

Yes, the use case is economic simulations. I’m starting to work together with Jacky Mallett who created the [ThreadNeedle](https://www.researchgate.net/publication/272752115_Threadneedle_An_Experimental_Tool_for_the_Simulation_and_Analysis_of_Fractional_Reserve_Banking_Systems) framework and we’re looking into the possibility to do something similar in Julia.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 17, 2021, 8:25am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/4 "2021-09-17T08:25:52Z")

</div>

[https://github.com/jackymallett/Threadneedle](https://github.com/jackymallett/Threadneedle) for other people looking through this thread, this seems to be the repo?

If the use case is financial modeling, I think you’re already kind of behind the curve. Looking through the [case studies on juliacomputing.com](https://juliacomputing.com/case-studies/):

- [Solvency II Compliance](https://juliacomputing.com/case-studies/aviva/)
  - One of Europe’s largest insurers is using Julia for Solvency II compliance

- [Financial Modeling at Atteson Research](https://juliacomputing.com/case-studies/atteson/)
- [Now-Casting Economics uses Julia to reduce macroeconomic modeling time from weeks to days](https://juliacomputing.com/case-studies/now-casting/)
- [Conning is using Julia in large scale Monte Carlo simulations for insurance risk assessment solutions](https://juliacomputing.com/case-studies/conning/)
  - They’re explicitly saying they chose julia for distributed parallel computing capabilities

- [The New York Federal Reserve](https://juliacomputing.com/case-studies/ny-fed/) also uses julia, they’ve put their code on Github as [DSGE.jl](https://frbny-dsge.github.io/DSGE.jl/stable/) and they showed a considerable [performance increase](https://frbny-dsge.github.io/DSGE.jl/stable/julia_forecasting/#Performance-Improvements-1) (those numbers are from julia 0.4.5, but as julia has only gotten faster, I think those speedups have also gone up - it would be interesting to know about how fast their modeling is nowadays).
- Thomas J. Sargent also held an [opening keynote for juliacon 2016](https://www.youtube.com/watch?v=KkKBwJkYgVk), titled “Quantitative Macroeconomics in Julia”.

So from what I can tell, there are already quite a few actors in that space using julia for that purpose, quite a few of which are also using this forum or are on [julialang.zulipchat.com](http://julialang.zulipchat.com). I sadly don’t know these people by name, as that is not my area of expertise, but I’ve linked this thread on zulip so maybe people with that expertise can chime in.

---

<div class="post-metadata">

**Author:** ![stefkuypers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefkuypers/32/19185_2.png) [@stefkuypers](https://discourse.julialang.org/u/stefkuypers)\
**Post date:** [September 17, 2021, 9:19am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/5 "2021-09-17T09:19:27Z")

</div>

Yes, that’s the repo.

Thanks a lot! We’ll look into it.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 17, 2021, 9:25am UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/6 "2021-09-17T09:25:57Z")

</div>

My best advise would be to just try things out and if any specific questions arise, you can ask them here on discourse, on the julialang zulip or on slack. We’re always happy to help 🙂

---

<div class="post-metadata">

**Author:** ![stefkuypers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefkuypers/32/19185_2.png) [@stefkuypers](https://discourse.julialang.org/u/stefkuypers)\
**Post date:** [September 17, 2021, 12:36pm UTC](https://discourse.julialang.org/t/how-well-does-julia-support-distributed-processing/68312/7 "2021-09-17T12:36:34Z")

</div>

Thanks!