# \#pmap

**URL:** https://discourse.julialang.org/tag/pmap/196.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Using pmap within a package](https://discourse.julialang.org/t/using-pmap-within-a-package/131627)

<div class="topic-metadata">

**Author:** [@veddox](https://discourse.julialang.org/u/veddox)\
**Replies:** 6\
**Last updated:** [August 16, 2025, 9:26am UTC](https://discourse.julialang.org/t/using-pmap-within-a-package/131627 "2025-08-16T09:26:40Z")

</div>

I’ve been trying to wrap my head around how to use pmap() properly within a package, but I’m still a bit confused and am starting to suspect my original plan may not work. For context: I’m working on a process-based eco…

---

## [Use break or continue statement with pmap](https://discourse.julialang.org/t/use-break-or-continue-statement-with-pmap/126425)

<div class="topic-metadata">

**Author:** [@Thomas\_Van\_Giel](https://discourse.julialang.org/u/Thomas_Van_Giel)\
**Replies:** 5\
**Last updated:** [March 2, 2025, 11:20pm UTC](https://discourse.julialang.org/t/use-break-or-continue-statement-with-pmap/126425 "2025-03-02T23:20:29Z")

</div>

Is there any alternative to continue or break when using a pmap from the distributed package? If I try the following minimal example: using Distributed as = \[i for i in 0:0.1:10\] status = pmap(as) do a if a \< 1 …

---

## [Lack of improvement from distributed pmap, understanding a simple example](https://discourse.julialang.org/t/lack-of-improvement-from-distributed-pmap-understanding-a-simple-example/121880)

<div class="topic-metadata">

**Author:** [@ma-y](https://discourse.julialang.org/u/ma-y)\
**Replies:** 6\
**Last updated:** [October 29, 2024, 2:25pm UTC](https://discourse.julialang.org/t/lack-of-improvement-from-distributed-pmap-understanding-a-simple-example/121880 "2024-10-29T14:25:31Z")

</div>

Dear all, I am relatively new to Julia and have spent the last weeks trying to parallelize my code, without much success. Therefore I would like to understand the following minimal example. My computer has 16 cores, an…

---

## [Manifest\_usage.toml/pidfile related error](https://discourse.julialang.org/t/manifest-usage-toml-pidfile-related-error/112585)

<div class="topic-metadata">

**Author:** [@Sid1](https://discourse.julialang.org/u/Sid1)\
**Replies:** 1\
**Last updated:** [October 3, 2024, 11:46am UTC](https://discourse.julialang.org/t/manifest-usage-toml-pidfile-related-error/112585 "2024-10-03T11:46:00Z")

</div>

I keep coming across this error (inconsistently) when I run my scripts on a HPC cluster: Activating project at \`/projects\` ERROR: LoadError: On worker 8: IOError: open("/user/.julia/logs/manifest\_usage.toml.pid", 194,…

---

## [Stopping pmap when one of the node hit an error](https://discourse.julialang.org/t/stopping-pmap-when-one-of-the-node-hit-an-error/109389)

<div class="topic-metadata">

**Author:** [@muuusso](https://discourse.julialang.org/u/muuusso)\
**Replies:** 0\
**Last updated:** [January 29, 2024, 11:05am UTC](https://discourse.julialang.org/t/stopping-pmap-when-one-of-the-node-hit-an-error/109389 "2024-01-29T11:05:36Z")

</div>

I’m using pmap on my cluster university to distribute computation between different nodes. I’d like to be able to stop all the workers when one of them fails/throw an error. I’ve been looking for a while and I couldn’t f…

---

## [Weird UndefRefError in pmap since 1.8](https://discourse.julialang.org/t/weird-undefreferror-in-pmap-since-1-8/91567)

<div class="topic-metadata">

**Author:** [@K\_B](https://discourse.julialang.org/u/K_B)\
**Replies:** 5\
**Last updated:** [November 25, 2023, 4:16am UTC](https://discourse.julialang.org/t/weird-undefreferror-in-pmap-since-1-8/91567 "2023-11-25T04:16:49Z")

</div>

Hey. I am kind of new to the language and my coding knowledge is very minimal. Therefore I am unable to solve (even understand) the problem I have right now. My code is kind of large and convoluted and I don’t know how …

---

## [Discarding pmap’s results (pforeach?)](https://discourse.julialang.org/t/discarding-pmap-s-results-pforeach/101452)

<div class="topic-metadata">

**Author:** [@lmtzx9h4qqnt](https://discourse.julialang.org/u/lmtzx9h4qqnt)\
**Replies:** 2\
**Last updated:** [July 10, 2023, 8:56pm UTC](https://discourse.julialang.org/t/discarding-pmap-s-results-pforeach/101452 "2023-07-10T20:56:46Z")

</div>

I was wondering what the best way to use pmap (or something equivalent) is when I just want to do some operations in parallel without collecting any results in memory. Silly example: julia\> pmap(x -\> println(x), 1:10) 1…

---

## [Unhandled task error using pmap](https://discourse.julialang.org/t/unhandled-task-error-using-pmap/87927)

<div class="topic-metadata">

**Author:** [@poopsilon](https://discourse.julialang.org/u/poopsilon)\
**Replies:** 1\
**Last updated:** [June 21, 2023, 1:55pm UTC](https://discourse.julialang.org/t/unhandled-task-error-using-pmap/87927 "2023-06-21T13:55:28Z")

</div>

Hi, I’m using pmap to distribute a computation to many workers on a cluster, pretty straightforward. Here I am using 1 node and 10 processors. Things seem to be going fine for most cases, but sometimes I get an error li…

---

## [Extend distributed pmap to mutliple nodes using SLURM](https://discourse.julialang.org/t/extend-distributed-pmap-to-mutliple-nodes-using-slurm/98263)

<div class="topic-metadata">

**Author:** [@psterzinger](https://discourse.julialang.org/u/psterzinger)\
**Replies:** 2\
**Last updated:** [May 3, 2023, 5:07pm UTC](https://discourse.julialang.org/t/extend-distributed-pmap-to-mutliple-nodes-using-slurm/98263 "2023-05-03T17:07:48Z")

</div>

I have a program that makes use of the parallelism offered by the Distributed’s pmap() function. I can run this on a cluster using one node and multiple cores. Now I would like to extend this to using multiple cores. Th…

---

## [Distributing parallel tasks/workers over multiple nodes using SLURM](https://discourse.julialang.org/t/distributing-parallel-tasks-workers-over-multiple-nodes-using-slurm/60574)

<div class="topic-metadata">

**Author:** [@schmuelinsky](https://discourse.julialang.org/u/schmuelinsky)\
**Replies:** 10\
**Last updated:** [October 25, 2022, 3:41am UTC](https://discourse.julialang.org/t/distributing-parallel-tasks-workers-over-multiple-nodes-using-slurm/60574 "2022-10-25T03:41:45Z")

</div>

I’m trying to compute a series of Monte Carlo (MC) integrations on an HPC using SLURM. Since the MC samples are all independent, my approach is to distribute them as parallel tasks using pmap(). Each MC sample is quite …

---

## [Write a function that stands up and shuts down workers?](https://discourse.julialang.org/t/write-a-function-that-stands-up-and-shuts-down-workers/89041)

<div class="topic-metadata">

**Author:** [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)\
**Replies:** 5\
**Last updated:** [October 22, 2022, 5:25am UTC](https://discourse.julialang.org/t/write-a-function-that-stands-up-and-shuts-down-workers/89041 "2022-10-22T05:25:19Z")

</div>

I have a utility in a package to run a batch of things. I’d like for that batch function to be able to take in an argument like n\_workers, then stand those workers up, distribute the work, and then remove those new worke…

---

## [How to do a nested parallelization of two nested functions?](https://discourse.julialang.org/t/how-to-do-a-nested-parallelization-of-two-nested-functions/88334)

<div class="topic-metadata">

**Author:** [@structural](https://discourse.julialang.org/u/structural)\
**Replies:** 13\
**Last updated:** [October 7, 2022, 11:33am UTC](https://discourse.julialang.org/t/how-to-do-a-nested-parallelization-of-two-nested-functions/88334 "2022-10-07T11:33:58Z")

</div>

Suppose I have two functions, with one nested inside the other. I have N processors for the computation. Is it possible to distribute the outer function over n1 processors and distribute each call of the inside function …

---

## [Strange error when passing a DataFrame to pmap](https://discourse.julialang.org/t/strange-error-when-passing-a-dataframe-to-pmap/88316)

<div class="topic-metadata">

**Author:** [@mb96](https://discourse.julialang.org/u/mb96)\
**Replies:** 1\
**Last updated:** [October 6, 2022, 8:10am UTC](https://discourse.julialang.org/t/strange-error-when-passing-a-dataframe-to-pmap/88316 "2022-10-06T08:10:06Z")

</div>

Hi there! I am having an error when parallelizing my code with the Distributed library from which I use pmap. The following code might seem a bit strange but it is a (as much as possible) simplification of what I was run…

---

## [Unhandled Task Error](https://discourse.julialang.org/t/unhandled-task-error/87749)

<div class="topic-metadata">

**Author:** [@jisutich](https://discourse.julialang.org/u/jisutich)\
**Replies:** 2\
**Last updated:** [September 24, 2022, 11:13pm UTC](https://discourse.julialang.org/t/unhandled-task-error/87749 "2022-09-24T23:13:34Z")

</div>

Hi, I am trying to use pmap to speed up my code. When I try to run a sample code with a for loop of just 10 cycles, I faced an unhandled task error. What does it mean? Personally I think the 10 cycles have finished …

---

## [Pmap suddenly after running for some time gives Load Error](https://discourse.julialang.org/t/pmap-suddenly-after-running-for-some-time-gives-load-error/87646)

<div class="topic-metadata">

**Author:** [@mb96](https://discourse.julialang.org/u/mb96)\
**Replies:** 0\
**Last updated:** [September 22, 2022, 2:08pm UTC](https://discourse.julialang.org/t/pmap-suddenly-after-running-for-some-time-gives-load-error/87646 "2022-09-22T14:08:46Z")

</div>

I am paralellizing, with the library Distributed (pmap method), calling the JuMP solver Ipopt across different agents (“utilities”). The results for each agent are then vertically concatenated into a DataFrame. The code …

---

## [Extracting elements from pmap using for loop](https://discourse.julialang.org/t/extracting-elements-from-pmap-using-for-loop/87516)

<div class="topic-metadata">

**Author:** [@mb96](https://discourse.julialang.org/u/mb96)\
**Replies:** 0\
**Last updated:** [September 20, 2022, 12:48pm UTC](https://discourse.julialang.org/t/extracting-elements-from-pmap-using-for-loop/87516 "2022-09-20T12:48:49Z")

</div>

I am confused about Julia behavior when I used pmap return object as an iterator. I will first show a chunk of code isomorphic to the malfunctioning code but which works and then show the malfunctioning code. Consider t…

---

## [Using Distributed computing in JULIA for UNet](https://discourse.julialang.org/t/using-distributed-computing-in-julia-for-unet/86483)

<div class="topic-metadata">

**Author:** [@momo1](https://discourse.julialang.org/u/momo1)\
**Replies:** 4\
**Last updated:** [August 31, 2022, 5:22am UTC](https://discourse.julialang.org/t/using-distributed-computing-in-julia-for-unet/86483 "2022-08-31T05:22:21Z")

</div>

Hi I have a UNet training for Machine Learning which works on my laptop (14 core) but its extremely slow. I’m trying to explore ways of speeding it up and came across Distributed library. So I’ve only added the follow…

---

## [Blend pmap with dynamic addprocs() rmprocs() on shared High Performance Computing Cluster](https://discourse.julialang.org/t/blend-pmap-with-dynamic-addprocs-rmprocs-on-shared-high-performance-computing-cluster/82702)

<div class="topic-metadata">

**Author:** [@philip](https://discourse.julialang.org/u/philip)\
**Replies:** 7\
**Last updated:** [June 17, 2022, 12:47am UTC](https://discourse.julialang.org/t/blend-pmap-with-dynamic-addprocs-rmprocs-on-shared-high-performance-computing-cluster/82702 "2022-06-17T00:47:45Z")

</div>

I have a “trivially parallel” computation to run. The function is loosely like this: function process\_one\_iteration(i::Int64, input\_directory::String) x = read\_input\_from\_directory(i, input\_directory::String) …

---

## [Using pmap: Do all workers have to be equally capable?](https://discourse.julialang.org/t/using-pmap-do-all-workers-have-to-be-equally-capable/74122)

<div class="topic-metadata">

**Author:** [@chelseas](https://discourse.julialang.org/u/chelseas)\
**Replies:** 5\
**Last updated:** [April 29, 2022, 12:18pm UTC](https://discourse.julialang.org/t/using-pmap-do-all-workers-have-to-be-equally-capable/74122 "2022-04-29T12:18:32Z")

</div>

I am using pmap with workers from many computers. Some of those computers have more resources than others ~ does pmap know this? And distribute tasks accordingly? e.g. if I have 10 workers on one machine vs 2 workers on …

---

## [Seed random number generator when using pmap](https://discourse.julialang.org/t/seed-random-number-generator-when-using-pmap/78973)

<div class="topic-metadata">

**Author:** [@jisutich](https://discourse.julialang.org/u/jisutich)\
**Replies:** 2\
**Last updated:** [April 4, 2022, 1:26pm UTC](https://discourse.julialang.org/t/seed-random-number-generator-when-using-pmap/78973 "2022-04-04T13:26:11Z")

</div>

Hi I am trying to use pmap to increase the speed of my code. And I try to seed the random generator in different processors, so I have a code like using Distributed addprocs(4) @everywhere using LinearAlgebra @everywh…

---

## [How to use distributed and pmap across GPU cores](https://discourse.julialang.org/t/how-to-use-distributed-and-pmap-across-gpu-cores/78852)

<div class="topic-metadata">

**Author:** [@Nadou407](https://discourse.julialang.org/u/Nadou407)\
**Replies:** 2\
**Last updated:** [April 1, 2022, 1:04pm UTC](https://discourse.julialang.org/t/how-to-use-distributed-and-pmap-across-gpu-cores/78852 "2022-04-01T13:04:25Z")

</div>

Hi all, I’m still new to Julia GPU and Julia in general so got quite confused about how to do parallel computation using a GPU. Any help would be greatly appreciated! My current code (a simple example) using 20 CPU cor…

---

## ["connection refused (ECONNREFUSED)" when trying to mpirun julia on multiple nodes](https://discourse.julialang.org/t/connection-refused-econnrefused-when-trying-to-mpirun-julia-on-multiple-nodes/78253)

<div class="topic-metadata">

**Author:** [@knkn1711](https://discourse.julialang.org/u/knkn1711)\
**Replies:** 2\
**Last updated:** [March 22, 2022, 5:54pm UTC](https://discourse.julialang.org/t/connection-refused-econnrefused-when-trying-to-mpirun-julia-on-multiple-nodes/78253 "2022-03-22T17:54:50Z")

</div>

I am trying to run my julia code on multiple nodes of a cluster, which uses Moab and Torque for the scheduler and resource manager. In an interactive session I run the following: qsub -A open -l walltime=1:00:00 -l node…

---

## [Pmapreduce with the keyword argument "dims"](https://discourse.julialang.org/t/pmapreduce-with-the-keyword-argument-dims/76256)

<div class="topic-metadata">

**Author:** [@dmetivie](https://discourse.julialang.org/u/dmetivie)\
**Replies:** 0\
**Last updated:** [February 11, 2022, 3:19pm UTC](https://discourse.julialang.org/t/pmapreduce-with-the-keyword-argument-dims/76256 "2022-02-11T15:19:29Z")

</div>

The function mapreduce(f, op, X; dims = dim) can be used, but it seems that the parallel analog pmapreduce from the package ParallelUtilities does not support the dims argument. How would one use simple parallel funct…

---

## [Is Pmap \_both\_ distributed and threaded?](https://discourse.julialang.org/t/is-pmap-both-distributed-and-threaded/73717)

<div class="topic-metadata">

**Author:** [@Chris\_Green](https://discourse.julialang.org/u/Chris_Green)\
**Replies:** 4\
**Last updated:** [December 28, 2021, 9:18pm UTC](https://discourse.julialang.org/t/is-pmap-both-distributed-and-threaded/73717 "2021-12-28T21:18:24Z")

</div>

My typical cluster setup is 20-60 48-core AWS nodes If I use pmap() to distribute work across them, will the cluster workers also distribute their chunk of the map operation among their threads?

---

## [When to use pmap vs Threads.@threads?](https://discourse.julialang.org/t/when-to-use-pmap-vs-threads-threads/72805)

<div class="topic-metadata">

**Author:** [@chelseas](https://discourse.julialang.org/u/chelseas)\
**Replies:** 11\
**Last updated:** [December 16, 2021, 8:28am UTC](https://discourse.julialang.org/t/when-to-use-pmap-vs-threads-threads/72805 "2021-12-16T08:28:31Z")

</div>

Some of my processes may take a minute, and others miliseconds, so load balancing seems like a good idea. Do both do this? Any other things I should know about when using pmap vs. @threads? I’ve tagged jump because I w…

---

## [Use pmap(f,x) when f takes multiple arguments?](https://discourse.julialang.org/t/use-pmap-f-x-when-f-takes-multiple-arguments/65761)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 3\
**Last updated:** [November 17, 2021, 9:44am UTC](https://discourse.julialang.org/t/use-pmap-f-x-when-f-takes-multiple-arguments/65761 "2021-11-17T09:44:22Z")

</div>

How can I call pmap(f,x) where f is a function that takes additional parameters? I have a function f(x,y) where y is always known and I want to run pmap on elements of x. Here is an example that works (without distribu…

---

## [Unexpected behaviour when interrupting pmap](https://discourse.julialang.org/t/unexpected-behaviour-when-interrupting-pmap/69668)

<div class="topic-metadata">

**Author:** [@albheim](https://discourse.julialang.org/u/albheim)\
**Replies:** 0\
**Last updated:** [October 13, 2021, 8:33am UTC](https://discourse.julialang.org/t/unexpected-behaviour-when-interrupting-pmap/69668 "2021-10-13T08:33:07Z")

</div>

I had a problem where I wanted to gracefully stop the execution of long running distributed jobs and return the data collected so far, and I ran into some behaviour for pmap that I found a bit odd. A small example of a …

---

## [Use of CachingPool's for a pmap that will be run many times](https://discourse.julialang.org/t/use-of-cachingpools-for-a-pmap-that-will-be-run-many-times/69462)

<div class="topic-metadata">

**Author:** [@tictaccat](https://discourse.julialang.org/u/tictaccat)\
**Replies:** 0\
**Last updated:** [October 9, 2021, 4:12am UTC](https://discourse.julialang.org/t/use-of-cachingpools-for-a-pmap-that-will-be-run-many-times/69462 "2021-10-09T04:12:06Z")

</div>

Hi! I’m writing a custom linear map using LinearMaps.jl whose matrix-vector product I would like to embarassingly parallelize. The point is that this linear map will be used tens of thousands of times during an iterative…

---

## [Poor speed gain using \`pmap\`](https://discourse.julialang.org/t/poor-speed-gain-using-pmap/65694)

<div class="topic-metadata">

**Author:** [@Rick](https://discourse.julialang.org/u/Rick)\
**Replies:** 17\
**Last updated:** [August 6, 2021, 8:01am UTC](https://discourse.julialang.org/t/poor-speed-gain-using-pmap/65694 "2021-08-06T08:01:49Z")

</div>

Hi, I am using the pmap function for parallelization. However, the speedup is not satisfactory. The speed gain is very far from linear. As an illustration, I count the time used when each simulation draws 10,000,000 ran…

---

## [Pmap using fewer workers than expected after some time](https://discourse.julialang.org/t/pmap-using-fewer-workers-than-expected-after-some-time/46715)

<div class="topic-metadata">

**Author:** [@Luapulu](https://discourse.julialang.org/u/Luapulu)\
**Replies:** 5\
**Last updated:** [February 10, 2021, 12:29pm UTC](https://discourse.julialang.org/t/pmap-using-fewer-workers-than-expected-after-some-time/46715 "2021-02-10T12:29:31Z")

</div>

I’m running pmap with 16 workers and batch\_size=1 to process hundreds of files. Each file takes over an hour to work through and the files are relatively homogeneous but don’t take exactly the same time. For the first 14…

[Next page](https://discourse.julialang.org/tag/pmap/196.md?match_all_tags=true&page=1&tags%5B%5D=pmap)