# Running a distributed calculation on JuliaHub

**URL:** <https://discourse.julialang.org/t/running-a-distributed-calculation-on-juliahub/69660>\
**Category:** New to Julia\
**Tags:** distributed, juliahub\
**Created:** [October 13, 2021, 6:56am UTC](https://discourse.julialang.org/t/running-a-distributed-calculation-on-juliahub/69660 "2021-10-13T06:56:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![AndreyO](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andreyo/32/26612_2.png) [@AndreyO](https://discourse.julialang.org/u/AndreyO)\
**Post date:** [October 13, 2021, 6:56am UTC](https://discourse.julialang.org/t/running-a-distributed-calculation-on-juliahub/69660/1 "2021-10-13T06:56:07Z")

</div>

Hi!

I’m trying to run a distributed nested loop on JuliaHub with 3-10 workers. The code evaluates a marginal likelihood function on a 3-d grid with the help of a distributed array:

```julia
logML_mat = Array{Float64}(undef, np, ngl, ngt, ngs)
logML_mat = distribute(logML_mat; dist=(1,1,1,nworkers()))
@sync @distributed for w in 1:nworkers()
    logML_mat_loc = localpart(logML_mat)
    li = localindices(logML_mat)
    for (ls,s) in enumerate(li[4])
        for p in 1:np
            for i in 1:ngl
                for j in 1:ngt  
                    logML_mat_loc[p,i,j,ls] = logML( [lambda_grid[i], theta_grid[j], psi_grid[s]] )                        
                end
            end
        end
    end
end

```

The code runs successfully, but after the outer (distributed) loop is finished, the following error occurs:

`LoadError: LoadError: LoadError: TaskFailedException nested task error: On worker 2: TaskFailedException nested task error: peer 3 is not connected to 2. Topology : master_worker`

What might be the case?

Best,  
Andrey

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [October 13, 2021, 3:56pm UTC](https://discourse.julialang.org/t/running-a-distributed-calculation-on-juliahub/69660/2 "2021-10-13T15:56:35Z")

</div>

This failure has to do with the communication topology of the cluster, which only allows direct communication between the head node and worker nodes. @tanmaykm is looking at enabling all-to-all communication and will post an update here after he looks into it.