Display progress in ThreadsX

General Usage
,
1

How can I display progress when using ThreadsX, with for example ProgressMeter?
Silly MWE, but imagine fun was a relatively expensive function:

using ThreadsX
fun(x) = (sleep(1); sin(x))
x = rand(100)
y = ThreadsX.map(fun, x) # show the progress somehow
2

There’s Transducers.withprogress which can be used as tcollect(withprogress(fun(a) for a in x); basesize = 1) given that you have corresponding https://github.com/JuliaLogging/ProgressLogging.jl monitors such as VS code and TerminalLoggers.jl.

Apparently I forgot to support withprogress in ThreadsX.jl. So you’d need to invoke Transducers.jl API directly ATM.

MWE:

julia> using Logging, TerminalLoggers

julia> global_logger(TerminalLogger());

julia> using Transducers

julia> function fun(x)
           t0 = time_ns()
           while time_ns() - t0 < 1e8
           end
           sin(x)
       end
fun (generic function with 1 method)

julia> x = rand(300);

julia> y = tcollect(withprogress(fun(a) for a in x); basesize = 1)

That said, the responsiveness of withprogress in threaded execution is not very nice. Depending on the actual run-time of fun, it may work fine, though.

3 Likes