How to modify nprocs with DistributedArrays inside a function

Hi,
I want to draw speed-up plots with Distributed and DistributedArrays.
I use Distributed.addprocs inside my benchmark function but I should also
use @everywhere using DistributedArrays that cannot (true ?) be called inside a function…
Any hints ?

1 Like

I found a semi-solution here https://stackoverflow.com/questions/47685536/how-do-you-load-a-module-everywhere-inside-a-function-in-julia/47691491

It is possible to use using inside a function with the expression:

    eval(macroexpand(Distributed,quote @everywhere using DistributedArrays end))

Not exactly pretty IMHO…

The following solution is almost the same, but arguably prettier:

julia> using Distributed

julia> function fun()
           addprocs(1)
           @eval @everywhere using Statistics
           @spawnat 2 mean([1,2,3])
       end
fun (generic function with 1 method)

julia> nprocs()
1

julia> fut = fun()
Future(2, 1, 5, nothing)

julia> fetch(fut)
2.0

julia> nprocs()
2
3 Likes