Solving equation within Distributed loop

General Usage
, ,
1

Hello, I am trying to solve an equation within a distributed loop. A MWE follows:

using Distributed

addprocs(2);
include("newfile.jl")

#Parameters:
par1 = 5;
par2 = 2;
@everywhere begin
    using CSV, DataFrames, BlackBoxOptim;
end
@everywhere mod_pars = Dict("par1"=>$par1,"par2"=>$par2);
@everywhere par_values = collect(1:10);
@distributed for iv = 1:10
    mod_pars["par1"] = par_values[iv];        
    myfun(mod_pars,iv);    
end

rmprocs(workers())

where newfile is just:

@everywhere g(mod_pars,x1,x2) = mod_pars["par1"]*((1.0 - x1)^2 + 100.0 * (x2 - x1^2)^2);

@everywhere function myfun(mod_pars,iv)
    f(x) = g(mod_pars,x[1],x[2]);
    res = bboptimize(f; SearchRange = (-5.0, 5.0), NumDimensions = 2);
    df_tmp = DataFrame();
    df_tmp[!,:values1] = ones(10)*mod_pars["par1"];
    df_tmp[!,:values2] = ones(10)*best_candidate(res)[1];
    CSV.write(joinpath("figures","data_"*string(iv)*".csv"),df_tmp);
end

This does not work with @distributed but does without and I am not sure why. Any suggestions?

Thank you very much!

2

I think you might be removing the processes before they do any work. The documentation indicates that @distributed returns immediately without waiting for the processes to complete and suggests @sync @distributed if you want to wait.

3

Thank you, that worked splendidly.

One question though. Does this mean that once a first machine is done with its iteration, it will wait until all other machines are done with theirs?

Thanks again.

4

Yes, I think that is what is meant by “The specified range is partitioned and locally executed across all workers.” Each worker process gets part of the range to work on, then @sync waits for all workers to finish.

1 Like