ANN: MPI.jl v0.10.0: new build process and CUDA-aware support

simonbyrne · August 16, 2019, 4:18am

I have just tagged a new version of MPI.jl. Though the user-facing interface is largely the same, there has been extensive work underneath to internally use the C API (instead of the Fortran one). As a result, the build process is much simpler (it no longer requires CMake or a Fortran compiler). Additionally, it also directly supports CUDA-aware MPI libraries, allowing CuArrays to be passed directly as buffers (thanks to Seyoon Ko).

I would greatly appreciate if people are able to try it out, especially with different clusters and MPI implementations.

johnh · August 16, 2019, 5:06am

Is it worth sharing this on the OpenMPI mailing list? I would guess you work closely with those guys anyway.

johnh · August 16, 2019, 5:09am

I just noticed this is in Julia at Scale!
Picture me doing a happy dance. Or maybe my Miata doing doughnuts (see avatar image).

kose-y · September 3, 2019, 3:20am

Thank you @simonbyrne! Here are some code examples using the new version of MPI.jl.

mcreel · September 4, 2019, 12:46am

Basic send Send, Recv! works fine on an ordinary Linux cluster with OpenMPI. Thanks!

rveltz · September 9, 2019, 4:35pm

Nice! Is there a way to partition a big CuArray into p parts like the slab decomposition?

kose-y · September 10, 2019, 12:10am

I’m working on it based on barche/MPIArrays.jl. I think it can be released some time this year.

rveltz · September 10, 2019, 5:55am

You may be interested in the python lib.

kose-y · September 10, 2019, 7:36am

@rveltz This looks interesting. Thanks for the reference!

johnh · September 13, 2019, 10:11am

I have access to a DGX-1 GPU system. I’m not sure how much time I can get on it.
If there are any tests which could be run for this package I could give it a try.

simonbyrne · September 13, 2019, 4:11pm

Do you know if it has a CUDA-aware MPI? If so, would be good to run the test suite with JULIA_PROJECT set to [pkgdir]/test/cudaenv.

samo · December 3, 2019, 5:54pm

@kose-y and @simonbyrne, a big thanks for making CUDA-aware MPI available to the Julia community! Unfortunately, when I tried CUDA-aware MPI with MPI.jl on two different systems, it failed in both cases. Could you have a look at this post where I reported the errors? It would be fantastic if I got it to work before the AGU conference this weekend…