# Help with performance tuning this dataframe aggregation

**URL:** <https://discourse.julialang.org/t/help-with-performance-tuning-this-dataframe-aggregation/15357>\
**Category:** Performance\
**Created:** [September 22, 2018, 10:04pm UTC](https://discourse.julialang.org/t/help-with-performance-tuning-this-dataframe-aggregation/15357 "2018-09-22T22:04:48Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![laborg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laborg/32/5474_2.png) [@laborg](https://discourse.julialang.org/u/laborg)\
**Post date:** [September 23, 2018, 1:17pm UTC](https://discourse.julialang.org/t/help-with-performance-tuning-this-dataframe-aggregation/15357/9 "2018-09-23T13:17:18Z")

</div>

Hi,

I think you are doing this a little bit to complicated. Especially the inner loops over columns `m` and `n` aren’t necessary.

Using Query.jl following solution is much faster:

```julia
function looptest7(data,loopsize)
	summary = DataFrame([Int64, Int64, Float64],[:cfg, :count, :ave],0)
	for i in 1:loopsize
		data.m .= fld.(data.x, i)
		for j in 1:loopsize
			cfg = i * j
			data.n .= fld.(data.y, j)

			x = @from i in data begin
				@group i.z by {i.m, i.n} into g
				@where length(g) == cfg
				@select minimum(g)
				@collect
			end

			push!(summary, [cfg length(x) mean(x)])
		end
	end
	return summary
end

julia> @btime looptest7(data,4)
  42.480 ms (184858 allocations: 13.03 MiB)

```

The not shared `looptest6` was a similar to `looptest5` but without using Query.jl and only using `groupby`. `looptest7` is twice as fast, because `groupby` on DataFrames isn’t typestable… (see: [Type of groupby(df,id) elements are Any](https://discourse.julialang.org/t/type-of-groupby-df-id-elements-are-any/9872))

---

_[View the full topic](https://discourse.julialang.org/t/help-with-performance-tuning-this-dataframe-aggregation/15357)._