# What is the real @time for an MPI program?

**URL:** <https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458>\
**Category:** Performance\
**Tags:** parallel\
**Created:** [April 23, 2019, 11:02pm UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458 "2019-04-23T23:02:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Carol](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carol/32/8140_2.png) [@Carol](https://discourse.julialang.org/u/Carol)\
**Post date:** [April 23, 2019, 11:02pm UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/1 "2019-04-23T23:02:58Z")

</div>

Hi, I have a question on timing an MPI code, because I want to test its performance.

For example, if I split my code into 10 parallel jobs using MPI, then the `@time` will return 10 timing records. BTW, I also try to use `start = time(); ... ;elapsed = time() - start` , but it still return 10 records as `@time` .

Is the real time for these 10 jobs the **maximum** , or the **sum**?

eg.

4.166182 seconds (436.84 k allocations: 21.515 MiB, 0.08% gc time)  
3.853497 seconds (436.84 k allocations: 21.515 MiB, 0.09% gc time)  
4.091061 seconds (436.84 k allocations: 21.515 MiB, 0.08% gc time)  
4.169676 seconds (436.84 k allocations: 21.515 MiB, 0.08% gc time)  
4.114676 seconds (436.84 k allocations: 21.515 MiB, 0.08% gc time)  
0.705804 seconds (436.84 k allocations: 21.515 MiB, 0.50% gc time)  
4.054897 seconds (436.84 k allocations: 21.515 MiB, 0.09% gc time)  
3.813533 seconds (436.84 k allocations: 21.515 MiB, 0.09% gc time)  
4.148482 seconds (436.84 k allocations: 21.515 MiB, 0.08% gc time)  
4.012410 seconds (436.84 k allocations: 21.515 MiB, 0.09% gc time)

Thanks!

---

<div class="post-metadata">

**Author:** ![JaredCrean2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jaredcrean2/32/3574_2.png) [@JaredCrean2](https://discourse.julialang.org/u/JaredCrean2)\
**Post date:** [April 24, 2019, 12:36am UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/2 "2019-04-24T00:36:05Z")

</div>

I’m not clear on the precisely how you are generating the output shown (a minimal example, or even some pseudo-code would help), but if you have a code like this:

```julia
# file tmp.jl
using MPI

@time begin
  MPI.Init()
   # do computation here
  MPI.finalize()
end

```

And then run `mpirun -np 10 julia ./tmp.jl`, then each of the 10 MPI processes will print the results of `@time` for that process (`@time` will not do any summing or averaging or anything like that).

The question of what is the “real” time does not have a well defined answer because the processes do not necessarily finish at the same time. You could say the “real” time is the time it takes the longest process to finish. You could also say the “real” time is the length of time from when the first process is created to when the last one finishes (keeping in mind that these might be different processes).

---

<div class="post-metadata">

**Author:** ![Carol](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carol/32/8140_2.png) [@Carol](https://discourse.julialang.org/u/Carol)\
**Post date:** [April 24, 2019, 3:52am UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/3 "2019-04-24T03:52:37Z")

</div>

Hi, I write a MPI dot prodect function. For example

```julia
function mpi_dot(a,b,n)
    MPI.Init()
    .......
    allsum = MPI.Reduce(local_sum,MPI.SUM, 0, comm)
    MPI.finalize()
end

```

And I want to test its speed by running `@time mpi_dot(a,b,n)`. Then I can compare it with non-parallel dot product.

But now I have multiple time records, and I don’t know which time I can use to do the comparison.  
Can I just the maximum time record?

---

<div class="post-metadata">

**Author:** ![JaredCrean2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jaredcrean2/32/3574_2.png) [@JaredCrean2](https://discourse.julialang.org/u/JaredCrean2)\
**Post date:** [April 24, 2019, 12:46pm UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/4 "2019-04-24T12:46:16Z")

</div>

Because the `Reduce` function is returning the result only on process 0, I would use the time from process 0.

---

<div class="post-metadata">

**Author:** ![Carol](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carol/32/8140_2.png) [@Carol](https://discourse.julialang.org/u/Carol)\
**Post date:** [May 9, 2019, 11:06pm UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/5 "2019-05-09T23:06:12Z")

</div>

Hi, I tried to print the rank id to identify the order of each process, then I can know which result is for process 0.

What surprise me is that the time of process 0 is not the maximum result among all time results.  
Could you please tell me why? Then I should still use the time for process 0?

Thanks

---

<div class="post-metadata">

**Author:** ![JaredCrean2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jaredcrean2/32/3574_2.png) [@JaredCrean2](https://discourse.julialang.org/u/JaredCrean2)\
**Post date:** [May 10, 2019, 1:07am UTC](https://discourse.julialang.org/t/what-is-the-real-time-for-an-mpi-program/23458/6 "2019-05-10T01:07:44Z")

</div>

I’m not sure why. It can depend on the order in which `mpirun` launches processes, which is not defined by the MPI standard.

I think the time on process 0 is a reasonable choice. The max time would also be a reasonable choice. I’m not sure one is significantly better than the other.