Hello!
I am getting started with Julia parallel computing and I successfully created a code that runs on different workers when launched from the Atom editor. However, I am facing some problems when running the same code from the terminal and I came out with a minimum working example which hopefully could explain the issue.
The basic algorithm is as follows:
using Distributed
addprocs(4)
@sync @distributed for idx=1:12
cd("current path") #change the current dir of each worker on the current one
println("$(idx)")
open("./results/$(idx).txt","a+") do fid
write(fid, "$(idx)\n")
end
end
This code is perfectly capable of printing the .txt files in the output director.
(side note: Does someone know if there is a way to directly make the workers act on the current folder instead of writing the cd in the for loop?)
I then launch the code from the command line with the following command:
nohup julia -p 2 ./code.jl > 1.out 2>&1 < /dev/null
The exact code I run from the command line does not need using Distributed (already contained in -p), does not need to add workers and to change the folder. Therefore, it will look like that:
@sync @distributed for idx=1:12
println("$(idx)")
open("./results/$(idx).txt","a+") do fid
write(fid, "$(idx)\n")
end
end
To my understanding, the problem seems to be connected to the macro @sync, which is not preventing the workers to open and write different files at the same time, forcing the stop of the execution. In fact, if I comment out the lines related to the writing of the files, the program correctly outputs the $(idx) in the log code.
Could someone tell me what it could be wrong with my approach?
Thank you.