Displaying Parallel Progress Bars

General Usage
5

In case this helps anyone else, I’ve hacked together a working version of this functionality using ProgressMeter.jl as a base. I’ve been using it for a while now, and it seems to work well enough. If anyone has suggestions, I would be happy to incorporate them. You can find the package here:

https://github.com/adamslc/ProgressMeter.jl

To use it, add some worker processes and define a work function:

addprocs(4)

@everywhere using ProgressMeter

@everywhere function test_function(x, p=nothing)
    @info("Running with value $x")

    @parallelprogress p "optional label " for i in 1:x
        sleep(0.1)
    end

    @info("Done")
    return x
end

Then it can either be run directly on the master process, or farmed out to the worker processes:

julia> test_function(100)
INFO: Running with value 100
optional label 100%|████████████████████████████████████| Time: 0:00:10
INFO: Done
100

julia> ProgressMeter.pmap(test_function, [50, 50, 100, 25, 10, 50])
2 tasks remaining to start
-------------------------------
optional label  40%|██████████████                      |  ETA: 0:00:04
optional label  80%|█████████████████████████████       |  ETA: 0:00:01
optional label  40%|██████████████                      |  ETA: 0:00:04
optional label  20%|███████                             |  ETA: 0:00:10

The @info macros allow you to print diagnostic information while you are still developing, and then easily run the function in parallel once everything is working.

4 Likes