# Some feedback while waiting for pmap

**URL:** https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970
**Category:** Julia at Scale
**Created:** [September 14, 2018, 6:27pm UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970 "2018-09-14T18:27:26Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![turtle](https://avatars.discourse-cdn.com/v4/letter/t/ecb155/32.png) [@turtle](https://discourse.julialang.org/u/turtle)
#### Post date: [September 14, 2018, 6:27pm UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/1 "2018-09-14T18:27:26Z")

</div>

I am running a long calculation distributed on several workers and several computers:  
`output = pmap( i->iter(i,parameters), 1:1000 )`

While waiting I would like to know which `i` is being processed.  
How can I get this simple info on master? (without writing files)

---

<div class="post-metadata">

### Author: ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)
#### Post date: [September 14, 2018, 8:02pm UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/2 "2018-09-14T20:02:41Z")

</div>

The poor man’s solution (aka my solution) was print statements inside the function being called.

---

<div class="post-metadata">

### Author: ![turtle](https://avatars.discourse-cdn.com/v4/letter/t/ecb155/32.png) [@turtle](https://discourse.julialang.org/u/turtle)
#### Post date: [September 14, 2018, 9:30pm UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/3 "2018-09-14T21:30:49Z")

</div>

Thanks.  
But how can I get rid of the “From worker:” part of the printed messages?  
Also I would like to see less than a thousand lines.

---

<div class="post-metadata">

### Author: ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)
#### Post date: [September 14, 2018, 10:14pm UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/4 "2018-09-14T22:14:34Z")

</div>

No idea. I think there is some work being done on ProgressMeter.jl to add functionality for `pmap`

> **[GitHub - timholy/ProgressMeter.jl: Progress meter for long-running computations](https://github.com/timholy/ProgressMeter.jl)**
>
> Progress meter for long-running computations. Contribute to timholy/ProgressMeter.jl development by creating an account on GitHub.

---

<div class="post-metadata">

### Author: ![turtle](https://avatars.discourse-cdn.com/v4/letter/t/ecb155/32.png) [@turtle](https://discourse.julialang.org/u/turtle)
#### Post date: [September 15, 2018, 8:29am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/5 "2018-09-15T08:29:14Z")

</div>

Thank you very much for pointing me to `ProgressMeter.jl`, it is definitely a useful package.

But unfortunately, `pmap` seems to be out of its focus:

`@showprogress 1 output=pmap( i->iter(i,parameters), 1:1000 )`

ERROR: LoadError: LoadError: ArgumentError: Final argument to @showprogress must be a for loop or comprehension.

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [September 16, 2018, 6:34am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/6 "2018-09-16T06:34:32Z")

</div>

You could set up a RemoteChannel and have workers query the channel before printing. The master node could then write a value to the channel indicating that you want a worker to print some info.

---

<div class="post-metadata">

### Author: ![turtle](https://avatars.discourse-cdn.com/v4/letter/t/ecb155/32.png) [@turtle](https://discourse.julialang.org/u/turtle)
#### Post date: [September 16, 2018, 7:19am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/7 "2018-09-16T07:19:38Z")

</div>

Thank you for the advice.  
As I have never used remote channels before,  
any short code example would be acknowledged.

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [September 16, 2018, 9:35am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/8 "2018-09-16T09:35:23Z")

</div>

```julia
using Distributed
addprocs(2)
const channel = RemoteChannel(()->Channel{Int}(1))

@async pmap(1:20) do i
    isready(channel) && println(i)
    sleep(3)
end

```

Then to activate printing `put!(channel, 1)` and to deactivate printing `take!(channel)`.

---

<div class="post-metadata">

### Author: ![turtle](https://avatars.discourse-cdn.com/v4/letter/t/ecb155/32.png) [@turtle](https://discourse.julialang.org/u/turtle)
#### Post date: [September 16, 2018, 10:13am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/9 "2018-09-16T10:13:04Z")

</div>

Clever code, thank you very much.

My remaining problem is the same as was with the “poor man’s solution”.  
When getting the feedback info using `println` I can see the results,  
but also the “From worker:” extra messages and 1000 separate lines.  
If I try to replace `println` by the function `print`, then nothing is printed.

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [September 16, 2018, 10:24am UTC](https://discourse.julialang.org/t/some-feedback-while-waiting-for-pmap/14970/10 "2018-09-16T10:24:03Z")

</div>

Well, you could have the worker put it’s latest iteration index and the master fetching it and doing the printing
