Right way to run external program in parallel?

I could not figure this out all day, until finally this answer saved me:

run(`bash -c """cd temp_dir && some_cmd"""`)

Is there a better way to do this?

In particular I want to run a program designed to log results sequentially each time it is run. Ie by writing/appending to files in its working directory. Currently I duplicate the directory, run separate instances, then combine the results. It works but this seems very hacky.

Have you considered writing a shell script?

  1. run external program
  2. after external program finish, then run julia program

Is there a reason to run both julia program and external program in parallel?

1 Like

I use

run(Cmd(`some_cmd some_input`; dir=temp_dir))

inside an asyncmap to run multiple instances of a program in parallel.


Its essentially the innerloop of my simulation. You can get some idea from my earlier question here:

Essentially I am running a stochastic game engine in parallel (ie, simultaneous games each week, multiple replications of each season), comparing the results to a baseline, modifying the skill ratings of the players… repeat until the combination of ratings produce “realistic” results.

But I think you are right that perhaps I can use GNU parallel for part of this. Eg: File I/O in gnu parallel - Stack Overflow