Hi there! I am having an error when parallelizing my code with the Distributed library from which I use pmap. The following code might seem a bit strange but it is a (as much as possible) simplification of what I was running.
The main issue is that the code works with map() function but not with pmap() function. I will first provide the code then the error.
using Distributed
for i in workers()
rmprocs(i)
end
addprocs(1);
using CSV
@everywhere using DataFrames
@everywhere using Printf
function GenData(x::Array{Float64,1}, hourly_utility::DataFrame)
@everywhere function utility_gen(util::Int64, hourly_utility::DataFrame)
for y in collect(2012:2020)
@printf("utility is %d and year is %d \n", util, y)
end
end
utilities = [195, 3046, 3265, 5416, 5860, 6452, 6455, 6909];
fit_results_hourly = pmap(u -> utility_gen(u, hourly_utility),
utilities)
end
hourly_utility = CSV.read("/users/utility_hour_for_Julia.csv", DataFrame);
GenData([2.18, 4.68, 0.01, -0.02, 1.28, 0.58], hourly_utility)
If I run it it will raise the following error:
However if I instead use map instead of pmap it will run.
Can someone please provide me with some help to understand why this is happening?
Thanks a lot!