# Displaying Parallel Progress Bars

**URL:** https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148
**Category:** General Usage
**Created:** [June 7, 2017, 8:47pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148 "2017-06-07T20:47:36Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![adamslc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamslc/32/3452_2.png) [@adamslc](https://discourse.julialang.org/u/adamslc)
#### Post date: [June 7, 2017, 8:47pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/1 "2017-06-07T20:47:36Z")

</div>

I need to run many simulations that will each take several hours to complete. To speed up the process, I’m going to run several of these simulations in parallel. The trouble is, I’m not very patient, and so I want some sort of progress indicator for each simulation.

I am aware of [PmapProgressMeter.jl](https://github.com/slundberg/PmapProgressMeter.jl), but it isn’t quite what I had in mind. I would prefer a series of progress bars that shows the progress for the currently running simulation on each processor, as well as the number of simulations remaining to start.

It would look something like this:

```julia
5 Tasks remaining to start
--------------------------
Worker 2: 28%|████████████ | ETA: 0:44:11
  .
  .
  .
Worker 5: 65%|█████████████████ | ETA: 1:26:17

```

Does anyone know of a package that can do this?

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [June 7, 2017, 8:49pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/2 "2017-06-07T20:49:06Z")

</div>

If you use Juno you can use its progress bar in a nested fashion.

---

<div class="post-metadata">

### Author: ![adamslc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamslc/32/3452_2.png) [@adamslc](https://discourse.julialang.org/u/adamslc)
#### Post date: [June 7, 2017, 9:49pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/3 "2017-06-07T21:49:27Z")

</div>

Thanks for the suggestion, but I’m working over `ssh` so I need a terminal based solution.

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [June 7, 2017, 10:02pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/4 "2017-06-07T22:02:53Z")

</div>

There’s a WIP PR for this kind of API to be in Base, but I’m not sure the solution you’re looking for exists quite yet:

[https://github.com/JuliaLang/Juleps/pull/30#issuecomment-301248423](https://github.com/JuliaLang/Juleps/pull/30#issuecomment-301248423)

---

<div class="post-metadata">

### Author: ![adamslc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamslc/32/3452_2.png) [@adamslc](https://discourse.julialang.org/u/adamslc)
#### Post date: [June 21, 2017, 6:45pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/5 "2017-06-21T18:45:24Z")

</div>

In case this helps anyone else, I’ve hacked together a working version of this functionality using [ProgressMeter.jl](https://github.com/timholy/ProgressMeter.jl) as a base. I’ve been using it for a while now, and it seems to work well enough. If anyone has suggestions, I would be happy to incorporate them. You can find the package here:

[https://github.com/adamslc/ProgressMeter.jl](https://github.com/adamslc/ProgressMeter.jl)

To use it, add some worker processes and define a work function:

```julia
addprocs(4)

@everywhere using ProgressMeter

@everywhere function test_function(x, p=nothing)
    @info("Running with value $x")

    @parallelprogress p "optional label " for i in 1:x
        sleep(0.1)
    end

    @info("Done")
    return x
end

```

Then it can either be run directly on the master process, or farmed out to the worker processes:

```julia
julia> test_function(100)
INFO: Running with value 100
optional label 100%|████████████████████████████████████| Time: 0:00:10
INFO: Done
100

julia> ProgressMeter.pmap(test_function, [50, 50, 100, 25, 10, 50])
2 tasks remaining to start
-------------------------------
optional label 40%|██████████████ | ETA: 0:00:04
optional label 80%|█████████████████████████████ | ETA: 0:00:01
optional label 40%|██████████████ | ETA: 0:00:04
optional label 20%|███████ | ETA: 0:00:10

```

The `@info` macros allow you to print diagnostic information while you are still developing, and then easily run the function in parallel once everything is working.

---

<div class="post-metadata">

### Author: ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)
#### Post date: [May 29, 2020, 9:20pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/6 "2020-05-29T21:20:52Z")

</div>

It’s been a while since this post. Is there now a registered package that provides this kind of parallel progress bar setup for pmap and `@distributed` processes?

> [@adamslc](#):
>
> ```julia
> julia> ProgressMeter.pmap(test_function, [50, 50, 100, 25, 10, 50])
> 2 tasks remaining to start
> -------------------------------
> optional label 40%|██████████████ | ETA: 0:00:04
> optional label 80%|█████████████████████████████ | ETA: 0:00:01
> optional label 40%|██████████████ | ETA: 0:00:04
> optional label 20%|███████ | ETA: 0:00:10
> 
> ```

---

<div class="post-metadata">

### Author: ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)
#### Post date: [March 27, 2022, 8:36pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/7 "2022-03-27T20:36:54Z")

</div>

I’ve been looking for something like this (multiple/nested progress) myself. ProgressLoggers kinda does it, but not cleanly and only with TerminalLoggers, it seems.

---

<div class="post-metadata">

### Author: ![yha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yha/32/3502_2.png) [@yha](https://discourse.julialang.org/u/yha)
#### Post date: [March 27, 2022, 9:14pm UTC](https://discourse.julialang.org/t/displaying-parallel-progress-bars/4148/8 "2022-03-27T21:14:33Z")

</div>

Back when Juno was still maintained, I wrote a simple package for following progress of `Distributed` workers, which could be displayed inside Juno plot pane. In combination with `TerminalLoggers` it can show a basic display of textual progress bars.  
[https://github.com/yha/ObservablePmap.jl](https://github.com/yha/ObservablePmap.jl)  
Unfortunately, it doesn’t work in vscode plot pane, but it can work inside a `Blink` window, like this:

```julia
using Distributed
using Blink: Window, body!
addprocs(...)

@everywhere using ObservablePmap
@everywhere using ProgressLogging
@everywhere using TerminalLoggers

summ, task = ologpmap( 'a':'e', 2:2:10; logger_f=TerminalLogger ) do c, x
    @withprogress name="Processing '$c'" for i=1:x
        sleep(rand())
        @logprogress i/x
    end
    @info "finished '$c'."
    x
end

html = map(x -> HTML("<pre>$x</pre>"), summ)

w = Window()
body!(w, html)

```

The downside is that you need a dedicated window for showing the progress. Also, on my Windows machine that `<pre>` tag doesn’t seem to respected in the Blink window for some reason - it uses a non-monospace font, so the progress bars don’t quite look right.
