Can't get right ranks when using non-blocking MPI.Irecv!()

New to Julia
1

I am trying to let rank 0 get other rank indexes, but I got different results from MPI.Irecv!() and MPI.Recv() , and result from MPI.Irecv!() seems odd:

import MPI

function main()
    MPI.Init()
    comm = MPI.COMM_WORLD
    size = MPI.Comm_size(comm) 
    rank = MPI.Comm_rank(comm) 
    if rank==0
        buf = 10
        mid = [10]
        for i in 1:size-1
            MPI.Recv!(mid, i, 0, comm)
            println("mid: $mid")
            buf = mid[1]
            println("buf: $buf")
        end
    else
        MPI.Isend([rank], 0, 0, comm)
    end
    MPI.Finalize()
end 
main()

Which output：

mid: [1]
buf: 1
mid: [2]
buf: 2
mid: [3]
buf: 3

If I replace MPI.Recv!(mid, i, 0, comm) by MPI.Irecv!(mid, i, 0, comm) , the output seemed not right:

mid: [10]
buf: 10
mid: [10]
buf: 10
mid: [10]
buf: 10

So my question is, why mid didn’t change when I use MPI.Irecv!(), and how to get the right rank number with MPI.Irecv!()?
Thanks for any suggestions and answers!

2

MPI.Irecv! is a non-blocking operation, that is, it returns immediately without waiting for the operation to complete. So by the time you call println(mid), data in mid has not been yet updated since the data has not been yet received.

Unlike its blocking variant (MPI.Recv!), MPI.Irecv! returns a “request”. At some point of your code, you need to wait for this request to complete, before you can operate on the received data. In your specific case, you want to do the following:

for i in 1:size-1
    req = MPI.Irecv!(mid, i, 0, comm)
    MPI.Wait!(req)  # wait for operation to complete
    println("mid: $mid")
    buf = mid[1]
    println("buf: $buf")
end

Note that all of this also applies to MPI.Isend, which is also non-blocking.

1 Like
3

Got it, thank you for always helping me!

4

Hi Polanco,
I still have a question about the piece of code, I would appreciate it if you could help me.
In the loop:

req = MPI.Irecv!(mid, i, 0, comm)
MPI.Wait!(req)  # wait for operation to complete
buf = mid[1]

It seems odd, I did it because when I try to get buf directly:

req = MPI.Irecv!([buf], i, 0, comm)

I couldn’t get the result I wanted. Do you have any suggestions? Thank you！

5

Hi again,

Note that buf is immutable (it’s an Int), which is why you can’t pass it directly to Irecv! as a buffer.

When you wrap it in an array, as in [buf], you’re passing Irecv! a single-element array. That function then replaces buf by the received value. Note that the array is being modified, not buf itself (which, again, is immutable). In your last bit of code, you’re losing the received information since the array is anonymous. That’s why you need to attach the array to a variable (like mid in your previous code).

2 Likes
6

Very clear, thank you!