# Problems using pmap(), and doubt about the number of workers/processes to use

**URL:** <https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510>\
**Category:** General Usage\
**Tags:** pmap\
**Created:** [February 6, 2019, 5:24pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510 "2019-02-06T17:24:29Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ale](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@ale](https://discourse.julialang.org/u/ale)\
**Post date:** [February 6, 2019, 5:24pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510/1 "2019-02-06T17:24:29Z")

</div>

Hello everyone,

I am using Julia 0.6.4.  
I am trying to use parallel computing in a Monte Carlo simulation program, by utilizing the `pmap()` function, so to apply the same function `my_function` to different inputs (which I will express in the following as `inputs` ) by means of different cores. So, the basic structure of my code will be like:

1. initialization
2. while … a certain condition  
outputs = pmap(my\_function, inputs);  
… additional operations…  
end
3. final calculations

Now, since my computer has only two physical cores, it should be recommended to use:  
`addprocs(1)`, so to have a total number of processes equal to 2. So my question is:  
will `pmap()` manage to use both process 1 and process 2, or maybe will it manage to use only process 2 consequently not taking advantage of process 1? I guess in the second case there would be no point to use `pmap()`.

I am wondering this because according to [https://docs.julialang.org/en/v0.6/manual/parallel-computing/](https://docs.julialang.org/en/v0.6/manual/parallel-computing/):

> The processes used by default for parallel operations are referred to as “workers”. When there is only one process, process 1 is considered a worker. Otherwise, workers are considered to be all processes other than process 1.

Thank you in advance for your help!

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [February 6, 2019, 8:28pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510/2 "2019-02-06T20:28:47Z")

</div>

> [@ale](#):
>
> `addprocs(1)` , so to have a total number of processes equal to 2. So my question is:  
> will `pmap()` manage to use both process 1 and process 2, or maybe will it manage to use only process 2 consequently not taking advantage of process 1?

If total processes is 2, then there is only 1 worker.  
If you want to use both cores as workers, then `addprocs(2)`  
You can confirm number of workers used by `pmap` with `nworkers()`

---

<div class="post-metadata">

**Author:** ![ale](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@ale](https://discourse.julialang.org/u/ale)\
**Post date:** [February 7, 2019, 4:15pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510/3 "2019-02-07T16:15:07Z")

</div>

@greg_plowman thank you very much for your feedback. However, I am still having problems implementing pmap, so I am not sure anymore the problem is only the number of processes/workers.  
In the following, I will provide more detailed information and rephrase my questions, hoping they are easier to answer.

So, as I said, I am using Julia 0.6.4, and my Windows 10 computer has 2 physical cores.  
I have a main script, where I call functions defined in other .jl files, for example:  
`include("monte_carlo_simulation.jl")`

The file “monte\_carlo\_simulation.jl” has is structured like this:

```julia
include("scenario_generator.jl")
include("optimization_function.jl")
function monte_carlo_simulation(input)
…body (in which, among the other things, I use the 2 just included functions)…
return output
end

```

That is, `monte_carlo_simulation()` uses other functions (all defined in the same project folder).  
Now, I am trying to implement parallel computing to speed up the simulations, by using` pmap()`.  
I have applied it in the following way. In the main script, I wrote:

```julia
addprocs(2)
@everywhere include("monte_carlo_simulation.jl")
…
ouput = pmap( monte_carlo_simulation, [input_1, input_2]);
output_1 = output[1];
output_2 = output[2];
…

```

1. **First question** :  
If I simply run the first two lines:

```julia
addprocs(2)
@everywhere include("monte_carlo_simulation.jl")

```

I get warnings saying:

> WARNING: replacing module JuMP.  
> WARNING: replacing module JuMP.  
> WARNING: replacing module Gurobi.  
> WARNING: replacing module Gurobi.

**What does it mean exactly? What could be the reason? And how to fix this?**  
Notice that JuMP and Gurobi are used in `optimization_function()`. In such a function, I wrote: `using JuMP, Gurobi` at the very beginning of the file.  
By the way, if I use `addprocs(1)` instead of `addprocs(2)`, the warning messages halves:

> WARNING: replacing module JuMP.  
> WARNING: replacing module Gurobi.

1. **Second question** :

The code runs with no further warnings or errors until just before running pmap(). But when I run also:  
`ouput = pmap(monte_carlo_simulation, [input_1, input_2]);`

I get the following error:

> **PyError (ccall(@pysym(:PyObject\_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw)) \<class ‘TypeError’\>**  
> **TypeError(“cannot serialize ‘\_io.TextIOWrapper’ object”,)**
> 
> in pmap at [base\distributed\pmap.jl:156](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in #pmap#213 at [base\distributed\pmap.jl:156](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in pmap at [base\distributed\pmap.jl:101](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in #pmap#203 at [base\distributed\pmap.jl:126](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in at [base&lt;missing\>](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in at [base&lt;missing\>](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in #async\_usemap#560 at [base\asyncmap.jl:103](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in wrap\_n\_exec\_twice at [base\asyncmap.jl:154](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in maptwice at [base\asyncmap.jl:178](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in foreach at [base\abstractarray.jl:1733](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)
> 
> in #575 at [base\asyncmap.jl:178](file:///C:/Users/username/AppData/Local/atom/app-1.34.0/resources/app.asar/static/index.html)

**Does anyone have any idea about what this is due to?**  
By the way, also with `addprocs(1)` instead of `addprocs(2)` the situation is the same.  
Thank you very much in advance!

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 7, 2019, 10:28pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510/4 "2019-02-07T22:28:37Z")

</div>

> [@ale](#):
>
> my Windows 10 computer has 2 physical cores

Why not just use `Threads.@threads`?