ProgressMeter while using DataFrame commands?

General Usage Performance
,
1

Hi,

I have a code chunk that uses DataFrame groupby() and combine() and that requires heavy computation for each group. I would like to see its progress such as out of n groups… how many groups have finished the computation currently.

I found a couple of potential options to do so such as ProgressMeter or @info.
However, it seems that many tutorials for those options are based on a for-loop approach or functions like map.

Below is a simple code that counts the number of observations by group gr.
I wonder if there is a way to display some sort of progress of the DataFrame combine() command in that code.

df = DataFrame(gr = rand(1:10, 100000))
@chain df begin
  groupby(:gr)
  combine(nrow)
end

Thanks!

3 Likes
2

@tkf can you help with this question.
tagging you after i saw your answer on https://discourse.julialang.org/t/help-with-progressmeter/31143t

3

I wonder if you could re-define the combine function but with ProgressMeter code (or using channels).