Parallelize simulations and print out results immediately when they are available

I have currently some code like this

function printsim(simulator, nsample)
    for i = 1:nsample 

As soon as each simulation is over, I print it to standard output, which I usually redirect to a file. In this way if the simulation is interrupted, I still have some results.

When I want to parallelize it, now I just start multiple Julia processes with a Python. But this sometimes create issues--

If I want to parallelize this inside Julia, how can I do it?

Have you read the guide on parallel computing? The 3rd section “Multi-Core or Distributed Processing” should do what you want.

AFAIK, there are still issues with IO and multithreaiding, but it should be no problem with parallel worker processes.

So there wouldn’t any issue if I print to stdout in every worker process? The output will not be scrambled?

I did a bit experiments, it seems to be fine to print from multiple process, but the output is like this

      From worker 2:	154407.10809850367 617409.1958756159 3.4875938817358254e6 
      From worker 2:	153981.47053866184 601490.2774611807 3.262175496086296e6

Is there any way to get rid of this “From worker”?

1 Like