Progress bar, or printing during execution, in Jupyter/IJulia?

[amca01]

I’m not sure whether this is a Julia question, or a Jupyter question, but anyway … I’m working with Julia in a Jupyter notebook (within VSCode on Windows). This is an environment I’m very happy with, and it suits me well. The trouble is that I have some functions that - for particular values of their parameters - take a long time to compute. And a progress bar, or even an occasional println statement during execution would be helpful. But in Jupyter (at least in VSCode), there is no output from the cell until it is completed.

Is there any nice way of getting information about execution of a function actually during its execution?

Thanks!

[Palli]

You can use Pluto and this in it, maybe it works in Jupyther too:

ProgressMeter in Pluto

I vaguely remember this being addressed in one of the many Pluto videos, but I am not sure. So my question is: can I use ProgressMeter, or something similar to show the progress of a loop inside a Pluto cell? Right now, there is no output at all from the code below. Ideally one would want to see a dynamic progress bar… e.g.: function test() @showprogress 1 "Computing..." for i in 1:50 sleep(0.1) end end

[stevengj]

ProgressMeter.jl works fine in Jupyter notebooks.

(You can also flush(stdout) to force println output to appear immediately.)

[photor]

I still expect a more interactive tool for Julia. Interact.jl is great, but cannot always work stably.

[amca01]

Thank you - I’d quite forgotten about flush(stdout). And I’ll try ProgressMeter.

[amca01]

Thank you - yes ProgressMeter works well on an indexed loop. The trouble is that I’m using a function from the Graphs package, which is known to be slow (requires a lot of computation), and I’m not sure how to meter it. I might ask on its GitHub page.

[stevengj]

@amca01

The trouble is that I’m using a function from the Graphs package, which is known to be slow (requires a lot of computation), and I’m not sure how to meter it. I might ask on its GitHub page.

See Progress meters for tasks with an unknown number of steps.

[amca01]

Actually it was easy in the end: rather than compute the eccentricities of all vertices of the graph G in one go as in eccentricity(G), I can do it one vertex at a time, building up the list as I go:

eccs = []
@showprogress for v in vertices(G)
      eccs = append!(eccs, [eccentricity(G,v)])
end

This is very slightly slower, but it does allow the use of a progress bar.

[stevengj]

@amca01

eccs = []
@showprogress for v in vertices(G)
      eccs = append!(eccs, [eccentricity(G,v)])
end

Offtopic, there are some problems here that I should point out:

1. If you compare to the eccentricities(g) implementation, you can see that you are calculating weights(G) over and over (or is this function just a “free” view?). Possibly better to compute distmx = weights(G) outside the loop, and then call eccentricity(G,v,distmx).

2. eccs = [] is an Any[] array, whereas concretely typed collections are generally preferred, e.g. eccs = eltype(distmx)[] where distmx = weights(G) as above.

3. append!(eccs, [eccentricity(G,v)]) allocates a 1-element array just to append it … better to do push!(eccs, eccentricity(G,v)). (You could also pre-allocate the array eccs and then just assign to it, since the length is known in advance, but probably not worth the trouble since eccentricity is expensive and push! is cheap. You could also use ProgressMeter.progress.map)

[amca01]

Many thanks indeed! That is excellent advice - you will see I’m somewhat of a Julia/Graphs newbie. (I’ve worked with Julia before, but I’ve done very little computations with graphs, in any language.) Using push instead of append! is something I guess I should have known … but bad habits from other languages die hard. And in fact most of my previous computations have been small enough that efficiency tweaks are unnecessary. I’m usually quite satisfied if something just works!

Thank you again - I very much appreciate your detailed and generous response.