Does anyone have a progress bar for pmap?

General Usage
1

I guess it’s been a while since anyone’s asked it, but does somebody have a working progress bar for pmap sitting around?

Related to:

1 Like
2

Best solution I could find is @slundberg’s package:

https://github.com/slundberg/PmapProgressMeter.jl

1 Like
3

Anyone thought of breathing life into this again? Would be great to have.

1 Like
4

ProgressMeter.jl has this now. You just need

@showprogress pmap(...)

For even more control, you could use the RemoteProgressWrapper I have in MuseInference.jl (registered), which wraps any arbitrary AbstractProgress and lets you call next! etc… on it from any worker. The code is here and an example of using it is here. Been meaning to do a PR into ProgressMeter.jl just haven’t had the time.

5 Likes
5

Adapted from the ProgressMeter.jl README (https://github.com/timholy/ProgressMeter.jl#tips-for-parallel-programming), here’s an example of using a progress bar with pmap where each pmapped element has its own batch of jobs (i.e., its own for-loop).

# n = length of `batches` * number of jobs per batch
progress = Progress(n)
channel = RemoteChannel(()->Channel{Bool}(), 1)

@async while take!(channel)
    next!(progress)
end

pmap(batch->begin
        for i in batch
            # ... some long computation here ...
            put!(channel, true) # trigger progress bar update
        end
    end, batches)

put!(channel, false) # tell printing task to finish

This is useful if you want to get progress updates from each completion of your “long computation”, rather than just progress updates over each element you’ve pmapped.