# Running the same function with multiple inputs using pmap

**URL:** https://discourse.julialang.org/t/running-the-same-function-with-multiple-inputs-using-pmap/24381
**Category:** General Usage
**Tags:** parallel
**Created:** [May 19, 2019, 4:12pm UTC](https://discourse.julialang.org/t/running-the-same-function-with-multiple-inputs-using-pmap/24381 "2019-05-19T16:12:03Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![magzpavz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/magzpavz/32/1822_2.png) [@magzpavz](https://discourse.julialang.org/u/magzpavz)
#### Post date: [May 19, 2019, 4:12pm UTC](https://discourse.julialang.org/t/running-the-same-function-with-multiple-inputs-using-pmap/24381/1 "2019-05-19T16:12:03Z")

</div>

I have a function which takes about 10 minutes to run. It is primarily in calling two external commands. I previously was using a bash script to run it. I need to run this function on several different inputs and would like to run them in parallel. Using `map` gives the expected results, but when I use `pmap`, it seems to be trying to parallelize the function itself, which I don’t want.

Further details:

The function definition looks like this:

```julia
@everywhere function getresult(x)  
    prefix = "si_$(x)"
    
    # string to control pw
    pw_in = """
        prefix='$prefix'
        """
    # string to control ph
    ph_in = """
          prefix='$prefix'
        """

    # run functions
    pw=open(`pw.x`,"r+") # create pw process
    print(pw,pw_in) # send it the control info
    close(pw.in)
    scf_out=read(pw,String) # waits until pw is done -- 3 minutes
    f_scf=open("./tmp/scfout_$prefix","w")
    write(f_scf,scfout) # write pw.x results to file for later
    close(f_scf)

    ph=open(`ph.x`,"r+") # create ph process
    print(ph,ph_in) # send it the control info
        # this process reads the results from pw based on prefix
    close(ph.in)
    phGout=read(ph,String) # waits until ph is done -- 7 minutes
    f_scf=open("./tmp/phGout_$prefix","w")
    write(f_scf,phGout) # write ph.x results to file for later
    close(f_scf)

    return(prefix)
end

```

The results are all written to files which are labeled by `prefix` which I define using the input variables. The process `ph.x` depends on the results that `pw.x` wrote, and knows where to look for them from its input string.

I try to run this function on multiple inputs using:

```julia
result = pmap(x->getresult(x), [1,2,3,4])

```

Julia has no problem, but the external processes fail because `ph.x` tries to run before `pw.x` has finished.

Some things I’ve tried:

- replacing `pmap` with `map` gives the expected result.
- running the function with different inputs on separate julia processes at the same time gives the expected results–it does not appear the external processes are interfering with each other.

Could you all help me figure out how to fix this? I am unfamiliar with parallel processing lingo which is making it hard to understand the docs. It feels like julia is trying to parallelize more than possible.

---

<div class="post-metadata">

### Author: ![zgornel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zgornel/32/217487_2.png) [@zgornel](https://discourse.julialang.org/u/zgornel)
#### Post date: [May 19, 2019, 9:50pm UTC](https://discourse.julialang.org/t/running-the-same-function-with-multiple-inputs-using-pmap/24381/2 "2019-05-19T21:50:15Z")

</div>

`pmap` will not parallelize the code im the function. On each core the function will perform as expected.

Are all `scfout_` output of the files created and if so, are the contents correct? What if you create two separate functions for the pw/ph sections and call pmap using each of them and thensame input vector?

---

<div class="post-metadata">

### Author: ![magzpavz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/magzpavz/32/1822_2.png) [@magzpavz](https://discourse.julialang.org/u/magzpavz)
#### Post date: [May 21, 2019, 1:00am UTC](https://discourse.julialang.org/t/running-the-same-function-with-multiple-inputs-using-pmap/24381/3 "2019-05-21T01:00:45Z")

</div>

Thank you for the suggestions. I tried writing two separate functions for each of pw and ph, but that did not help.

I did fix my problem, however, by writing the control string to a file and having the process read from that rather than passing the string directly. I don’t know why that fixed it, though, or why I never saw problems using plain `map`.
