# Will Distributed.jl support Infiniband?

**URL:** https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899
**Category:** General Usage
**Tags:** parallel, cluster
**Created:** [September 7, 2022, 4:01pm UTC](https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899 "2022-09-07T16:01:13Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![deltaeecs](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deltaeecs/32/21295_2.png) [@deltaeecs](https://discourse.julialang.org/u/deltaeecs)
#### Post date: [September 7, 2022, 4:01pm UTC](https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899/1 "2022-09-07T16:01:13Z")

</div>

I use the iterative solvers on a slurm cluster using DistributedArrays and I have to communicate data between workers every iteration, which is really slow because no InfiniBand support (10~200times slower than in-node data transfer).

Sadly, I don’t know about the InfiniBand thing until all the program is ok. I find the bottleneck of communication these days while optimizing the codes and then the administrator of cluster asked me whether my codes use InfiniBand to send message or not. It’s too late.

So, if there is any plan to add InfiniBand support for Distributed.jl? Or shall I just move all my code to [PartitionedArrays.jl](https://github.com/fverdugo/PartitionedArrays.jl) and  
[MPI.jl](https://github.com/JuliaParallel/MPI.jl))?

---

<div class="post-metadata">

### Author: ![deltaeecs](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deltaeecs/32/21295_2.png) [@deltaeecs](https://discourse.julialang.org/u/deltaeecs)
#### Post date: [September 8, 2022, 12:17am UTC](https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899/2 "2022-09-08T00:17:26Z")

</div>

up

---

<div class="post-metadata">

### Author: ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [September 8, 2022, 1:44am UTC](https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899/3 "2022-09-08T01:44:03Z")

</div>

I guess you mean Infiniband…  
Have a look at this thread: [Custom transport for Distributed.jl to utilize Infiniband and avoid MPI?](https://discourse.julialang.org/t/custom-transport-for-distributed-jl-to-utilize-infiniband-and-avoid-mpi/74097)

Because Distributed.jl is part of Julia, can you create an issue (feature request) at [Issues · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues?q=is%3Aissue+is%3Aopen) ?

---

<div class="post-metadata">

### Author: ![deltaeecs](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deltaeecs/32/21295_2.png) [@deltaeecs](https://discourse.julialang.org/u/deltaeecs)
#### Post date: [September 8, 2022, 2:08am UTC](https://discourse.julialang.org/t/will-distributed-jl-support-infiniband/86899/4 "2022-09-08T02:08:46Z")

</div>

Yes, I have noticed the discussion about [UCX.jl](https://github.com/JuliaParallel/UCX.jl), but there isn’t any documents about it and I’m not familiar with UCX at all. I wish I can do some contribute to the package. But for now, I think I’d better to switch my code to MPI to match the schedule. 🤒
