# \[WIP\] MPISort.jl - Distributed MPI Sorting Algorithms. Suggestions welcome!
**URL:** https://discourse.julialang.org/t/wip-mpisort-jl-distributed-mpi-sorting-algorithms-suggestions-welcome/88730
**Category:** Package Announcements
**Tags:** package, announcement, sort, mpi, distributed
**Created:** [October 14, 2022, 1:56pm UTC](https://discourse.julialang.org/t/wip-mpisort-jl-distributed-mpi-sorting-algorithms-suggestions-welcome/88730 "2022-10-14T13:56:18Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@anicusan](https://discourse.julialang.org/u/anicusan)
#### Post date: [October 14, 2022, 1:56pm UTC](https://discourse.julialang.org/t/wip-mpisort-jl-distributed-mpi-sorting-algorithms-suggestions-welcome/88730/1 "2022-10-14T13:56:18Z")
Hi all,
It was with _great horror_ that I discovered how non-trivial sorting N elements across P clusters _without any one cluster being able to hold all elements at once_ was. Or maybe I’m just thick. Anyways, I only found two such open-source algorithms written in not-so-friendly C++ and Charm++ and surprisingly few papers on the subject.
So here’s [`MPISort.jl`](https://github.com/anicusan/MPISort.jl), a Julia package offering `mpisort!` as an interface for such distributed sorting methods.
One algorithm is included at the moment: sampling with interpolated histograms, or `SIHSort` (pronounced _sigh_ sort, like anything MPI-related), optimised for minimum inter-rank communication and memory footprint. Features:
- **Does not require that distributed data fits into the memory of a single node**. No IO either.
- Works for any comparison-based data, with additional optimisations for numeric elements.
- Optimised for minimum MPI communication; can use Julia threads on each shared-memory node.
- The node-local arrays may have different sizes; sorting will try to balance number of elements held by each MPI rank.
- Works with any `AbstractVector`, including accelerators such as GPUs (this needs further testing). Julia type-inference and optimisations do wonders.
- Implements the standard Julia `sort!` API, and naturally works for custom data, comparisons, orderings, etc.
I really want to thank the [MPI.jl](https://github.com/JuliaParallel/MPI.jl) authors for making MPI such a joy to use in Julia; here’s to hopefully more great work in the Julia distributed ecosystem!
I am looking for any feedback on the library design, code, algorithm before I register the package. All suggestions are highly appreciated.
Best wishes,
Leonard
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [October 14, 2022, 1:59pm UTC](https://discourse.julialang.org/t/wip-mpisort-jl-distributed-mpi-sorting-algorithms-suggestions-welcome/88730/2 "2022-10-14T13:59:36Z")
Is this actually faster than a disk based sort? I would think that a disk based merge sort would pretty much always be faster since disks tend to have higher speed than networks, and you are going to be bottle-necked by IO speed.
---
### Author:  [@anicusan](https://discourse.julialang.org/u/anicusan)
#### Post date: [October 14, 2022, 2:24pm UTC](https://discourse.julialang.org/t/wip-mpisort-jl-distributed-mpi-sorting-algorithms-suggestions-welcome/88730/3 "2022-10-14T14:24:34Z")
The problem `SIHSort` solves is slightly different: I _want_ to have all my data in memory, spread across 10-1000s of nodes in a supercomputing clusters. The initial driver for this was a large particle simulation that can be split on 100 clusters, but it can’t all fit on a single one; in this case, sorting redistributes particles across nodes - this needs to be done relatively often, and so constant saving to disk is not desirable. Also, supercomputers almost always have extremely fast [interconnects](https://www.nvidia.com/en-gb/networking/products/infiniband/) with MPI implementations optimised for their hardware; I’m not sure if IO will be faster in that case.
That said, out-of-core sorting on a single machine was definitely considered as a future addition to `MPISort.jl`, where a lot of on-disk data must be sorted on a single computer with not enough RAM to hold all of it at once - hence the general interface `mpisort!` that may employ different algorithms.