# Is there a MPI-based map package?

**URL:** https://discourse.julialang.org/t/is-there-a-mpi-based-map-package/45087
**Category:** Numerics
**Tags:** hpc, mpi, distributed
**Created:** [August 17, 2020, 8:52am UTC](https://discourse.julialang.org/t/is-there-a-mpi-based-map-package/45087 "2020-08-17T08:52:15Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![astrojhgu](https://avatars.discourse-cdn.com/v4/letter/a/a9adbd/32.png) [@astrojhgu](https://discourse.julialang.org/u/astrojhgu)
#### Post date: [August 17, 2020, 8:52am UTC](https://discourse.julialang.org/t/is-there-a-mpi-based-map-package/45087/1 "2020-08-17T08:52:15Z")

</div>

Is there such a package providing a function like?

```julia
function mpi_map(Function, Array, MPI_COMM),

```

which bechaviours like

```julia
result=map(some_input_array, comm) do x
    some_operation(x)
done

```

I wrote one myself based on send/recv. A manager process listens to all worker process and assign jobs. However, which is very specified only to some certain problems. Is there package that provides more general ones?

[This](https://github.com/astrojhgu/MPIMap) is my own implementation.

---

<div class="post-metadata">

### Author: ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)
#### Post date: [August 17, 2020, 9:27am UTC](https://discourse.julialang.org/t/is-there-a-mpi-based-map-package/45087/2 "2020-08-17T09:27:09Z")

</div>

Not exactly what you want, but there’s [MPICLusterManagers.jl](https://github.com/JuliaParallel/MPIClusterManagers.jl) that has `@mpi_do` that does eg.:

```julia
@mpi_do manager begin
  using MPI
  comm=MPI.COMM_WORLD
  println("Hello world, I am $(MPI.Comm_rank(comm)) of $(MPI.Comm_size(comm))")
end

```

In this case the code is executed only on the workers. Perhaps this might be used for communications?

---

<div class="post-metadata">

### Author: ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [August 18, 2020, 4:14am UTC](https://discourse.julialang.org/t/is-there-a-mpi-based-map-package/45087/3 "2020-08-18T04:14:08Z")

</div>

MPIClusterManagers.jl has more than that, I think it has exactly what OP is asking for, you just need to use `Distributed.pmap`, e.g:

```julia
julia> using MPI, MPIClusterManagers, Distributed

julia> addprocs(MPIManager(np=4))

julia> result = pmap(1:10) do x
           MPI.Comm_rank(MPI.COMM_WORLD)
       end
10-element Array{Int64,1}:
 0
 1
 3
 2
 3
 0
 1
 2
 3
 0

```

Note you can also use the `MPIClusterManagers.start_main_loop` way of initializing MPI which lets you use `mpiexec -n` with a script.