# The ultimate guide to distributed computing
**URL:** https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867
**Category:** Julia at Scale
**Tags:** parallel, cluster, distributed
**Created:** [June 22, 2020, 2:52pm UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867 "2020-06-22T14:52:10Z")
**Posts on this page:** 5
**Page:** 3
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [March 12, 2021, 5:10pm UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867/41 "2021-03-12T17:10:09Z")
@Shazman the repository contains a minimum example that you can download and run. You can modify the example to your needs, but this is off-topic here.
---
### Author:  [@bdc](https://discourse.julialang.org/u/bdc)
#### Post date: [March 28, 2021, 6:28pm UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867/42 "2021-03-28T18:28:13Z")
@stephenll There is a possible use case: if you are running computation in parallel where the individual computation times vary significantly.
From the [@threads documentation](https://docs.julialang.org/en/v1/manual/multi-threading/#The-@threads-Macro):
> The iteration space is split among the threads, after which each thread writes its thread ID to its assigned locations
So basically, when running your parallel computation with `Threads.@threads` each computation is assigned to a thread a priori whereas using `pmap` will assign a computation to one of the workers a soon when it is available. E.g. If you have one computation that takes an order of magnitude longer than the other ones, using `Threads.@threads` will be running the long one and the thread’s additional assigned tasks, whereas with `pmap` on worker will work on the long computation and the other workers will deal with the faster ones.
I’ve added an example below. One thing that I have not considered is the possible slowdown due to the additional overhead, as the significance of this depends highly on the application you’re dealing with.
```julia
┌ Warning: running threaded
└ @ Main ~/Desktop/parallelcompare.jl:29
#= /Users/bart/Desktop/parallelcompare.jl:30 =# @benchmark(threaded_tasks()) = Trial(55.044 s)
┌ Warning: running distrubuted
└ @ Main ~/Desktop/parallelcompare.jl:31
#= /Users/bart/Desktop/parallelcompare.jl:32 =# @benchmark(distributed_task()) = Trial(18.013 s)
```
```Julia
using Distributed
using BenchmarkTools
@everywhere begin
using Logging
Logging.disable_logging(LogLevel(0))
function task(id::Int; duration=1)
@info "starting task $(id) on $(Threads.threadid()) (duration: $(duration))"
sleep(duration)
@info "finished task $(id) on $(Threads.threadid())"
end
end
function threaded_tasks(n=10)
@info "running on $(Threads.nthreads()) threads"
Threads.@threads for id in 1:10
task(id, duration=id)
end
end
function distributed_task(n=10)
@info "running on $(length(Distributed.workers())) workers"
pmap(x->task(x, duration=x), 1:n)
end
function main()
@warn "running threaded"
@show @benchmark threaded_tasks()
@warn "running distrubuted"
@show @benchmark distributed_task()
end
main()
```
---
### Author:  [@Elmo](https://discourse.julialang.org/u/Elmo)
#### Post date: [June 20, 2021, 7:19pm UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867/43 "2021-06-20T19:19:22Z")
Just chiming in, check out this package: [https://github.com/LCSB-BioCore/DistributedData.jl](https://github.com/LCSB-BioCore/DistributedData.jl), it is pretty awesome and makes lots of distributed computing tasks very straightforward… Potentially a good inclusion in the ultimate guide 🙂
---
### Author:  [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [June 21, 2021, 12:27am UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867/44 "2021-06-21T00:27:07Z")
There’s ThreadPools.jl to handle the varying work case.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [June 21, 2021, 11:02am UTC](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867/45 "2021-06-21T11:02:41Z")
PRs are welcome in the repository to add links to these packages. The more centralized are these resources the better for new users.
[Previous page](https://discourse.julialang.org/t/the-ultimate-guide-to-distributed-computing/41867.md?page=2)