# Right way to run external program in parallel?

**URL:** <https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490>\
**Category:** New to Julia\
**Tags:** parallel, filesystem, run, io\
**Created:** [January 31, 2024, 3:49am UTC](https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490 "2024-01-31T03:49:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [January 31, 2024, 3:49am UTC](https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490/1 "2024-01-31T03:49:01Z")

</div>

I could not figure this out all day, until finally this answer saved me:

```julia
run(`bash -c """cd temp_dir && some_cmd"""`)

```

> <https://stackoverflow.com/questions/43342132/how-to-change-cwd-for-a-command-executed-in-julia/68610667#68610667>

Is there a better way to do this?

In particular I want to run a program designed to log results sequentially each time it is run. Ie by writing/appending to files in its working directory. Currently I duplicate the directory, run separate instances, then combine the results. It works but this seems very hacky.

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [January 31, 2024, 3:57am UTC](https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490/2 "2024-01-31T03:57:15Z")

</div>

Have you considered writing a shell script?

1. run external program
2. after external program finish, then run julia program

Is there a reason to run both julia program and external program in parallel?

---

<div class="post-metadata">

**Author:** ![nhz2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhz2/32/44428_2.png) [@nhz2](https://discourse.julialang.org/u/nhz2)\
**Post date:** [January 31, 2024, 4:09am UTC](https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490/3 "2024-01-31T04:09:54Z")

</div>

I use

```julia
run(Cmd(`some_cmd some_input`; dir=temp_dir))

```

inside an `asyncmap` to run multiple instances of a program in parallel.

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [January 31, 2024, 4:21am UTC](https://discourse.julialang.org/t/right-way-to-run-external-program-in-parallel/109490/4 "2024-01-31T04:21:21Z")

</div>

Its essentially the innerloop of my simulation. You can get some idea from my earlier question here:

> [@What is the right way to store/record the results of this nested simulation?](https://discourse.julialang.org/t/what-is-the-right-way-to-store-record-the-results-of-this-nested-simulation/107639):
>
> I wrote this after seeing the responses to [my last question](https://discourse.julialang.org/t/how-to-best-preallocate-nested-tuples-of-dataframes-to-store-simulation-results/107543/33). From the responses I can tell my problem was not understood. What this simulation does is play a rudimentary “game” between pairs of two teams once per “week” and record the player results and league standings. Since the outcome is stochastic this is repeated multiple times. In actuality, the ratings/parameters of the players would change each simulation step and the output would be compared to other stats, but I left that out for no…

Essentially I am running a stochastic game engine in parallel (ie, simultaneous games each week, multiple replications of each season), comparing the results to a baseline, modifying the skill ratings of the players… repeat until the combination of ratings produce “realistic” results.

But I think you are right that perhaps I can use GNU parallel for part of this. Eg: [File I/O in gnu parallel - Stack Overflow](https://stackoverflow.com/questions/19952211/file-i-o-in-gnu-parallel)