# \#pmap

**URL:** https://discourse.julialang.org/tag/pmap/196.md?page=1

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 2

---

## [Pmap using fewer workers than expected after some time](https://discourse.julialang.org/t/pmap-using-fewer-workers-than-expected-after-some-time/46715)

<div class="topic-metadata">

**Author:** [@Luapulu](https://discourse.julialang.org/u/Luapulu)\
**Replies:** 5\
**Last updated:** [February 10, 2021, 12:29pm UTC](https://discourse.julialang.org/t/pmap-using-fewer-workers-than-expected-after-some-time/46715 "2021-02-10T12:29:31Z")

</div>

I’m running pmap with 16 workers and batch\_size=1 to process hundreds of files. Each file takes over an hour to work through and the files are relatively homogeneous but don’t take exactly the same time. For the first 14…

---

## [Scoping is too confusing with custom package and modules to use \`pmap\`](https://discourse.julialang.org/t/scoping-is-too-confusing-with-custom-package-and-modules-to-use-pmap/53383)

<div class="topic-metadata">

**Author:** [@iHany](https://discourse.julialang.org/u/iHany)\
**Replies:** 3\
**Last updated:** [January 16, 2021, 12:37pm UTC](https://discourse.julialang.org/t/scoping-is-too-confusing-with-custom-package-and-modules-to-use-pmap/53383 "2021-01-16T12:37:18Z")

</div>

I saw the previous post about pmap. However, the answers do not work for me. Here’s my example. What am I missing? Note that MyPackage is generated by PkgTemplates. src/MyPackage.jl module MyPackage using Reexport…

---

## [Dataframes with groupby in parallel](https://discourse.julialang.org/t/dataframes-with-groupby-in-parallel/47186)

<div class="topic-metadata">

**Author:** [@bicepjai](https://discourse.julialang.org/u/bicepjai)\
**Replies:** 1\
**Last updated:** [October 1, 2020, 4:39am UTC](https://discourse.julialang.org/t/dataframes-with-groupby-in-parallel/47186 "2020-10-01T04:39:06Z")

</div>

I have a dataframe of size (2450225, 117) Just using pandas and multiprocessing module, gives me the results in 34 seconds. I have panda code below. I am trying to achieve the same with julia and hoping this would be a …

---

## [PMAP Throws Exception on ProcessExitedException](https://discourse.julialang.org/t/pmap-throws-exception-on-processexitedexception/45667)

<div class="topic-metadata">

**Author:** [@Edward\_Wall](https://discourse.julialang.org/u/Edward_Wall)\
**Replies:** 1\
**Last updated:** [August 28, 2020, 9:13am UTC](https://discourse.julialang.org/t/pmap-throws-exception-on-processexitedexception/45667 "2020-08-28T09:13:40Z")

</div>

A short question. Is there some way to use pmap such that the following doesn’t throw an exception and crash? I have a distributed system, where a process might use too much memory and be killed by the os. This seems to …

---

## [Measuring and storing execution time in a variable](https://discourse.julialang.org/t/measuring-and-storing-execution-time-in-a-variable/45236)

<div class="topic-metadata">

**Author:** [@trathi](https://discourse.julialang.org/u/trathi)\
**Replies:** 2\
**Last updated:** [August 20, 2020, 4:50am UTC](https://discourse.julialang.org/t/measuring-and-storing-execution-time-in-a-variable/45236 "2020-08-20T04:50:03Z")

</div>

I want to measure the time from the moment the code enters pmap and exit it. Also, I want to store it in a variable. So is there a way I can store the output of @time or is there a julia package that does it more efficie…

---

## [Parallel computing using Optim](https://discourse.julialang.org/t/parallel-computing-using-optim/41948)

<div class="topic-metadata">

**Author:** [@Joris\_Pinkse](https://discourse.julialang.org/u/Joris_Pinkse)\
**Replies:** 5\
**Last updated:** [June 25, 2020, 3:55pm UTC](https://discourse.julialang.org/t/parallel-computing-using-optim/41948 "2020-06-25T15:55:49Z")

</div>

Hello, I’m running the program below on a 32 cpu/64 thread system without much of anything else running on it. If I use anything beyond 16 cores then the execution time in the second run is effectively flat. This is t…

---

## [Why is the parallel map so slow?](https://discourse.julialang.org/t/why-is-the-parallel-map-so-slow/39250)

<div class="topic-metadata">

**Author:** [@Tofi](https://discourse.julialang.org/u/Tofi)\
**Replies:** 2\
**Last updated:** [May 10, 2020, 11:49pm UTC](https://discourse.julialang.org/t/why-is-the-parallel-map-so-slow/39250 "2020-05-10T23:49:57Z")

</div>

I have an array a defined as: julia\> a = collect(1:10000) I can map a squaring function over the array and time the execution as following: julia\> @time map((x)-\>x^2, a) On my machine, the line above runs in 0.05 sec…

---

## [Why Pmap() skipping a combination in the function arguments](https://discourse.julialang.org/t/why-pmap-skipping-a-combination-in-the-function-arguments/38152)

<div class="topic-metadata">

**Author:** [@Saptarshi\_Ghosh](https://discourse.julialang.org/u/Saptarshi_Ghosh)\
**Replies:** 1\
**Last updated:** [April 25, 2020, 12:12pm UTC](https://discourse.julialang.org/t/why-pmap-skipping-a-combination-in-the-function-arguments/38152 "2020-04-25T12:12:13Z")

</div>

Hi! I am new to Julia and wondering about a simple implementation of the julia pmap function. Here is the sample code using Distributed; addprocs(6) #adding workers io = open("myfile.txt", "w"); write(io, "This is a …

---

## [Is there something similar to R's pmap in Julia?](https://discourse.julialang.org/t/is-there-something-similar-to-rs-pmap-in-julia/37055)

<div class="topic-metadata">

**Author:** [@ameresv](https://discourse.julialang.org/u/ameresv)\
**Replies:** 7\
**Last updated:** [April 6, 2020, 6:22am UTC](https://discourse.julialang.org/t/is-there-something-similar-to-rs-pmap-in-julia/37055 "2020-04-06T06:22:39Z")

</div>

Hi! I am new with Julia (being an R-guy) I have a question: Is something similar to pmap here in Julia? Why I am asking this, because I have this dataframe: m = DataFrame( x = \[1,2,3\], y = \[\[1,2,3\],\[4,5,6\],\[7,8…

---

## [Passing constructed closures to child processes](https://discourse.julialang.org/t/passing-constructed-closures-to-child-processes/34723)

<div class="topic-metadata">

**Author:** [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Replies:** 3\
**Last updated:** [February 20, 2020, 9:02pm UTC](https://discourse.julialang.org/t/passing-constructed-closures-to-child-processes/34723 "2020-02-20T21:02:04Z")

</div>

I am performing a distributed calculation via pmap. For context, I have some expensive model which I need to differentiate, and for this I’ve chosen Zygote. Part of the process is defining an “adjoint” for pmap. I have f…

---

## [Trying to understand communication between workers in simple version of pmap](https://discourse.julialang.org/t/trying-to-understand-communication-between-workers-in-simple-version-of-pmap/32531)

<div class="topic-metadata">

**Author:** [@Bruno\_Amorim](https://discourse.julialang.org/u/Bruno_Amorim)\
**Replies:** 4\
**Last updated:** [January 3, 2020, 11:39am UTC](https://discourse.julialang.org/t/trying-to-understand-communication-between-workers-in-simple-version-of-pmap/32531 "2020-01-03T11:39:00Z")

</div>

I am trying to understand how different workers can communicate in Julia. In the documentation of previous versions of Julia (\<= v0.6) in the Parallel Computing section, a simple pmap function was given as an example. T…

---

## [Pmap throwing BoundsErrors when accessing SharedArray](https://discourse.julialang.org/t/pmap-throwing-boundserrors-when-accessing-sharedarray/27652)

<div class="topic-metadata">

**Author:** [@DocDave](https://discourse.julialang.org/u/DocDave)\
**Replies:** 0\
**Last updated:** [August 17, 2019, 12:21pm UTC](https://discourse.julialang.org/t/pmap-throwing-boundserrors-when-accessing-sharedarray/27652 "2019-08-17T12:21:29Z")

</div>

A code which I am writing is heavily numerical in nature, but ultimately it calculates a value for each point of a 4D array according to the values within multiple separate 4D arrays. Consequently, now that I have a work…

---

## [Pmap use of processor cores](https://discourse.julialang.org/t/pmap-use-of-processor-cores/25121)

<div class="topic-metadata">

**Author:** [@abx](https://discourse.julialang.org/u/abx)\
**Replies:** 13\
**Last updated:** [June 12, 2019, 3:06pm UTC](https://discourse.julialang.org/t/pmap-use-of-processor-cores/25121 "2019-06-12T15:06:26Z")

</div>

I have a simple question about pmap and I can’t find a clear answer anywhere. When I have a series of pmap’d processes and let’s say, few of them are very heavy and require most of the resources, are the heavy tasks abl…

---

## [Issue with nested pmap](https://discourse.julialang.org/t/issue-with-nested-pmap/25129)

<div class="topic-metadata">

**Author:** [@abx](https://discourse.julialang.org/u/abx)\
**Replies:** 2\
**Last updated:** [June 11, 2019, 1:55pm UTC](https://discourse.julialang.org/t/issue-with-nested-pmap/25129 "2019-06-11T13:55:03Z")

</div>

Hello, I have this script running in Julia 1.1. If I run it withing a single process (no workers added) everything works fine. If I add process (e.g. addprocs(3)) nothing gets run and the execution hangs indefinitely u…

---

## [Generate Random Symmetric Matrices with pmap](https://discourse.julialang.org/t/generate-random-symmetric-matrices-with-pmap/24706)

<div class="topic-metadata">

**Author:** [@VincentTam](https://discourse.julialang.org/u/VincentTam)\
**Replies:** 2\
**Last updated:** [May 28, 2019, 7:39pm UTC](https://discourse.julialang.org/t/generate-random-symmetric-matrices-with-pmap/24706 "2019-05-28T19:39:02Z")

</div>

I would like to call Symmetric() inside an @everywhere function, and use pmap to generate an array of Symmetric matrices, which are half of the sizeof normal ones, but I received an UndefVarError. using LinearAlgebra, D…

---

## [Problems using pmap(), and doubt about the number of workers/processes to use](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510)

<div class="topic-metadata">

**Author:** [@ale](https://discourse.julialang.org/u/ale)\
**Replies:** 3\
**Last updated:** [February 7, 2019, 10:28pm UTC](https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510 "2019-02-07T22:28:37Z")

</div>

Hello everyone, I am using Julia 0.6.4. I am trying to use parallel computing in a Monte Carlo simulation program, by utilizing the pmap() function, so to apply the same function my\_function to different inputs (which …

---

## [Pmap with complex structs](https://discourse.julialang.org/t/pmap-with-complex-structs/18289)

<div class="topic-metadata">

**Author:** [@squaregoldfish](https://discourse.julialang.org/u/squaregoldfish)\
**Replies:** 7\
**Last updated:** [December 6, 2018, 5:59pm UTC](https://discourse.julialang.org/t/pmap-with-complex-structs/18289 "2018-12-06T17:59:04Z")

</div>

I have a large array of a complex mutable struct type. The whole array is processed iteratively, but each element can be processed independently of the others in a given iteration. The number of iterations required to co…

---

## [Behavior of worker pool in pmap](https://discourse.julialang.org/t/behavior-of-worker-pool-in-pmap/17882)

<div class="topic-metadata">

**Author:** [@swishmas](https://discourse.julialang.org/u/swishmas)\
**Replies:** 2\
**Last updated:** [November 25, 2018, 6:39pm UTC](https://discourse.julialang.org/t/behavior-of-worker-pool-in-pmap/17882 "2018-11-25T18:39:34Z")

</div>

Hi everyone, I’ve been coding in julia for a while, but I haven’t used any of the parallelization features beyond the Distributed package. My experience with parallelization is mainly with OpenMP in Fortran. I wanted to…

---

## [pmap usage pmap(xxxx,i) where i is large than nprocs](https://discourse.julialang.org/t/pmap-usage-pmap-xxxx-i-where-i-is-large-than-nprocs/11470)

<div class="topic-metadata">

**Author:** [@ersoacc](https://discourse.julialang.org/u/ersoacc)\
**Replies:** 0\
**Last updated:** [June 6, 2018, 1:27pm UTC](https://discourse.julialang.org/t/pmap-usage-pmap-xxxx-i-where-i-is-large-than-nprocs/11470 "2018-06-06T13:27:51Z")

</div>

Hello. I use pmap to calculate Jacobian matrix by multi-colored finite differences. My Jacobians are large and sparse. Thus, there are approximately 18 colors. Following is the line that computes all colors(colums). J…

---

## [Pmap error with closure](https://discourse.julialang.org/t/pmap-error-with-closure/10691)

<div class="topic-metadata">

**Author:** [@EthanAnderes](https://discourse.julialang.org/u/EthanAnderes)\
**Replies:** 7\
**Last updated:** [May 6, 2018, 2:01pm UTC](https://discourse.julialang.org/t/pmap-error-with-closure/10691 "2018-05-06T14:01:01Z")

</div>

I’ve been getting an error having to do with pmap on closures that I don’t understand. Anyone know what is going on? addprocs(2) function gen\_f(a::T) where T f = x -\> T return f end f = gen\_f(1f0) f(1f0) # …

---

## [Requesting idle workers to speed up unbalanced processes with pmap](https://discourse.julialang.org/t/requesting-idle-workers-to-speed-up-unbalanced-processes-with-pmap/9838)

<div class="topic-metadata">

**Author:** [@bchiem](https://discourse.julialang.org/u/bchiem)\
**Replies:** 9\
**Last updated:** [March 21, 2018, 8:12pm UTC](https://discourse.julialang.org/t/requesting-idle-workers-to-speed-up-unbalanced-processes-with-pmap/9838 "2018-03-21T20:12:38Z")

</div>

Dear Julia users, I am currently coding an algorithm that performs the same task on several pieces of a dataset in parallel using pmap, and then gathers the results in a SharedArray. Due to the exploratory nature of the…

---

## [Nested pmap() functions don not work as I expect](https://discourse.julialang.org/t/nested-pmap-functions-don-not-work-as-i-expect/9789)

<div class="topic-metadata">

**Author:** [@ersoacc](https://discourse.julialang.org/u/ersoacc)\
**Replies:** 3\
**Last updated:** [March 19, 2018, 12:36pm UTC](https://discourse.julialang.org/t/nested-pmap-functions-don-not-work-as-i-expect/9789 "2018-03-19T12:36:45Z")

</div>

Hello everyone. I am using pmap function to calculate and sum a function with different arguments such as: function gparal(nsubs) Gs = pmap(i -\> ASPINS1Sub(i, ...), 1:nsubs) return sum(G…

---

## [Clarification on pmap master-worker model](https://discourse.julialang.org/t/clarification-on-pmap-master-worker-model/7824)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 3\
**Last updated:** [December 18, 2017, 3:17am UTC](https://discourse.julialang.org/t/clarification-on-pmap-master-worker-model/7824 "2017-12-18T03:17:14Z")

</div>

In the master-worker model with pmap, what is the correct way of checking for parallel vs. serial execution? if nprocs() \> 1 # do task in parallel else # fallback to serial execution end or if nprocs() \> 2 # do …

[Previous page](https://discourse.julialang.org/tag/pmap/196.md?match_all_tags=true&tags%5B%5D=pmap)
