DataFrames operation scales badly

Dataframes group by are not ootimised. See Group-by performance benchmarks and recommendations for some background