Parallelize simulations and print out results immediately when they are available

New to Julia
1

I have currently some code like this

function printsim(simulator, nsample)
    for i = 1:nsample 
        println(simulation(simulator))
    end
end

As soon as each simulation is over, I print it to standard output, which I usually redirect to a file. In this way if the simulation is interrupted, I still have some results.

When I want to parallelize it, now I just start multiple Julia processes with a Python. But this sometimes create issues-- https://github.com/JuliaLang/julia/issues/31953

If I want to parallelize this inside Julia, how can I do it?

2

Have you read the guide on parallel computing? The 3rd section “Multi-Core or Distributed Processing” should do what you want.

https://docs.julialang.org/en/v1/manual/parallel-computing/index.html

AFAIK, there are still issues with IO and multithreaiding, but it should be no problem with parallel worker processes.

3

So there wouldn’t any issue if I print to stdout in every worker process? The output will not be scrambled?

4

I did a bit experiments, it seems to be fine to print from multiple process, but the output is like this

      From worker 2:	154407.10809850367 617409.1958756159 3.4875938817358254e6 
      From worker 2:	153981.47053866184 601490.2774611807 3.262175496086296e6

Is there any way to get rid of this “From worker”?

5
1 Like