# Is it possible to pass a struct in MPI.jl?

**URL:** <https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137>\
**Category:** Julia at Scale\
**Tags:** mpi, mpijl\
**Created:** [October 21, 2021, 9:45am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137 "2021-10-21T09:45:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![giancarloantonucci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giancarloantonucci/32/202551_2.png) [@giancarloantonucci](https://discourse.julialang.org/u/giancarloantonucci)\
**Post date:** [October 21, 2021, 9:45am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/1 "2021-10-21T09:45:17Z")

</div>

I have something like

```julia
struct Struct
   a::Vector
   b::Vector
   i::Integer
end

mystruct = Struct(rand(10), rand(5), 1)

```

How can I pass `mystruct` around with MPI.jl? If it’s not possible as it is, what’s the easiest workaround?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 21, 2021, 10:51am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/2 "2021-10-21T10:51:40Z")

</div>

IIRC, you’re struct must be `isbitstype`, which yours isn’t. First of all, all your field types are abstract since `Integer` is abstract and you didn’t specify the type parameters of `Vector`. This is bad for performance even before thinking about parallelization with MPI.

However, even if you would write your struct as

```julia
struct S
    a::Vector{Float64}
    b::Vector{Float64}
    i::Int64
end

```

it wouldn’t be `isbitstype`, because `Vector` doesn’t (statically) indicate the length of the vector.

If you know the length of the vectors, though, you can replace `Vector` by `SVector` from [StaticArrays.jl](https://github.com/JuliaArrays/StaticArrays.jl) to get

```julia
julia> struct StaticS
           a::SVector{10, Float64}
           b::SVector{10, Float64}
           i::Int64
       end

julia> isbitstype(StaticS)
true

```

Objects of this type can then be passed around via e.g. `MPI.Send`.

**A working example:**

```julia
using MPI
using StaticArrays
using Random

# must be isbitstype so we're using SVector here
struct Particle
    x::Float32
    y::Float32
    z::Float32
    velocity::Float32
    name::SVector{10,Char}
    mass::Float64
end

function Particle()
    return Particle(
        rand(Float32),
        rand(Float32),
        rand(Float32),
        rand(Float32),
        collect(randstring(10)),
        rand(),
    )
end

function main()
    # init MPI as always ...
    MPI.Init()
    rank = MPI.Comm_rank(MPI.COMM_WORLD)
    world_sz = MPI.Comm_size(MPI.COMM_WORLD)

    # send a vector of Particles
    N = 5
    if rank == 0
        println("Rank 0: Sending...")
        ps = [Particle() for _ in 1:N]
        for p in ps
            println(p)
        end
        println()
        # automatically calls create_struct under the hood
        MPI.Send(ps, 1, 0, MPI.COMM_WORLD)
    else
        recvbuf = Vector{Particle}(undef, N)
        MPI.Recv!(recvbuf, 0, 0, MPI.COMM_WORLD)
        println("Rank 1: Receiving...")
        for p in recvbuf
            println(p)
        end
    end

    return MPI.Finalize()
end

main()

```

Possible output:

```nohighlight
Rank 0: Sending...
Particle(0.46500897f0, 0.5270344f0, 0.9695773f0, 0.5295187f0, ['m', 'Y', 'L', 'o', 'k', 'Q', 'J', 'l', '9', 'P'], 0.02892147340795659)
Particle(0.28375614f0, 0.866768f0, 0.06535137f0, 0.059826612f0, ['W', 'E', 'C', 'd', 'A', 'v', 'q', '7', 'e', '1'], 0.1901452287781975)
Particle(0.22925854f0, 0.23061156f0, 0.3626809f0, 0.8808433f0, ['M', '8', 'a', '9', 'a', 'r', '1', '5', 'W', 'o'], 0.25132079245344663)
Particle(0.17365086f0, 0.510172f0, 0.20950961f0, 0.29564202f0, ['2', 'f', '0', '2', 't', 'i', 'n', '2', 'e', 'c'], 0.5372436257786044)
Particle(0.72283494f0, 0.3763957f0, 0.6775358f0, 0.9669912f0, ['N', 'd', '1', 'q', 'b', 'q', 'P', 't', '7', 'c'], 0.8375645365266158)

Rank 1: Receiving...
Particle(0.46500897f0, 0.5270344f0, 0.9695773f0, 0.5295187f0, ['m', 'Y', 'L', 'o', 'k', 'Q', 'J', 'l', '9', 'P'], 0.02892147340795659)
Particle(0.28375614f0, 0.866768f0, 0.06535137f0, 0.059826612f0, ['W', 'E', 'C', 'd', 'A', 'v', 'q', '7', 'e', '1'], 0.1901452287781975)
Particle(0.22925854f0, 0.23061156f0, 0.3626809f0, 0.8808433f0, ['M', '8', 'a', '9', 'a', 'r', '1', '5', 'W', 'o'], 0.25132079245344663)
Particle(0.17365086f0, 0.510172f0, 0.20950961f0, 0.29564202f0, ['2', 'f', '0', '2', 't', 'i', 'n', '2', 'e', 'c'], 0.5372436257786044)
Particle(0.72283494f0, 0.3763957f0, 0.6775358f0, 0.9669912f0, ['N', 'd', '1', 'q', 'b', 'q', 'P', 't', '7', 'c'], 0.8375645365266158)

```

---

<div class="post-metadata">

**Author:** ![giancarloantonucci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giancarloantonucci/32/202551_2.png) [@giancarloantonucci](https://discourse.julialang.org/u/giancarloantonucci)\
**Post date:** [October 21, 2021, 11:07am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/3 "2021-10-21T11:07:51Z")

</div>

Thank you for the detailed answer. I already knew that concrete parametrisation substantially improves performance, but I do understand why you might have wanted to clarify it. In fact, my struct example is more like

```julia
struct S{a_T, b_T}
    a::Vector{a_T}
    b::Vector{b_T}
    i::Int64
end

```

where both `a_T` and `b_T` are `isbitstype`, which is somewhat equivalent to your case. So, the second part of your answer is really what I was looking for. Also thank you for that working example.

What if the size of those vectors can’t be set a priori (e.g. they might collect the result of an adaptive ODE solver)? Is there a workaround in that case?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 21, 2021, 11:19am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/4 "2021-10-21T11:19:44Z")

</div>

> [@giancarloantonucci](#):
>
> What if the size of those vectors can’t be set a priori (e.g. they might collect the result of an adaptive ODE solver)? Is there a workaround in that case?

Depends on what precisely _a priori_ means. It is enough if the length is known when creating an object of type `S`, in which case one could get away with

```julia
julia> struct S{a_T, b_T, L}
           a::SVector{L, a_T}
           b::SVector{L, b_T}
           i::Int64
       end

julia> isbitstype(S{Float64, Float64, 10})
true

```

Does that suffice for your usecase?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 21, 2021, 11:22am UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/5 "2021-10-21T11:22:31Z")

</div>

Another option might be to make the static vectors large enough (upper bound) and adding an extra field `length` which indicates the “dynamic” length of the vector content.

If the vectors really need to be dynamically sized, I don’t know a solution from the top of my head.

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [October 21, 2021, 6:50pm UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/6 "2021-10-21T18:50:48Z")

</div>

The problem is that MPI generally expects both sides of the communication to know how much data is being communicated. So you can either communicate the lengths separately, then communicate the arrays individually, or you can use the lowercase MPI.send/MPI.recv functions, which will serialize the data.

---

<div class="post-metadata">

**Author:** ![giancarloantonucci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giancarloantonucci/32/202551_2.png) [@giancarloantonucci](https://discourse.julialang.org/u/giancarloantonucci)\
**Post date:** [October 22, 2021, 12:45pm UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/7 "2021-10-22T12:45:34Z")

</div>

> [@simonbyrne](#):
>
> or you can use the lowercase MPI.send/MPI.recv functions, which will serialize the data.

How do these functions (capitalised vs lowercase) differ? Or, better, what does serialisation mean? I’ve checked the MPI.jl documentation, but I guess I’m still a newbie in terms of MPI.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 22, 2021, 1:42pm UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/8 "2021-10-22T13:42:12Z")

</div>

> [@giancarloantonucci](#):
>
> Or, better, what does serialisation mean?

Serialization isn’t something MPI specific. See

[https://docs.julialang.org/en/v1/stdlib/Serialization/#Serialization](https://docs.julialang.org/en/v1/stdlib/Serialization/#Serialization)

and

> <https://github.com/JuliaParallel/MPI.jl/blob/6fdc2719ba3d97ee7a3eba43339e1a0cacf32d7f/src/MPI.jl#L9>

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [October 22, 2021, 4:13pm UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/9 "2021-10-22T16:13:46Z")

</div>

Serialization lets you write arbitrary data to a buffer, see [Serialization · The Julia Language](https://docs.julialang.org/en/v1/stdlib/Serialization/)

The lowercase `MPI.send` will serialize the data then call `MPI.Send` (`MPI.isend` is the non-blocking version of `MPI.send`); `MPI.recv` will probe the message to find out how long it is, allocate an appropriate buffer, call `MPI.Recv`, then deserialize the buffer and return the object.

These are naturally slower as the copy data and are not type stable, there are some benchmarks in the [MPI.jl paper](https://proceedings.juliacon.org/papers/10.21105/jcon.00068)

---

<div class="post-metadata">

**Author:** ![giancarloantonucci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giancarloantonucci/32/202551_2.png) [@giancarloantonucci](https://discourse.julialang.org/u/giancarloantonucci)\
**Post date:** [October 22, 2021, 4:23pm UTC](https://discourse.julialang.org/t/is-it-possible-to-pass-a-struct-in-mpi-jl/70137/10 "2021-10-22T16:23:47Z")

</div>

Thank you both. That was illuminating.