Save each simulation result on master node during parameter scanning

Specific Domains Julia at Scale
1

Hello !

I want to write code that stores data for a huge amount of simulation.
I want to save data for each experiment, and the data storage location is the master node.
Below is an example code, do you have any better ideas?

# TEST CODE

using Distributed

addprocs() # bring all local threads
addprocs([("###@####", :auto)]; dir="/home/####", exename="julia")

@everywhere using CSV
@everywhere using DataFrames
@everywhere function f()
    sleep(rand(0:10))
    return DataFrame(rand(10^3,10^2),:auto) # <- example data
end


mkpath("test_data")
n = 100000
asyncmap(1:1000000) do i # <- Really many simulations
    k = @fetch f()
    CSV.write("test_data/$i.csv",k) # save data on master node
    k = nothing
end