# Groupreduce of DataFrame with empty group

**URL:** <https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599>\
**Category:** General Usage\
**Created:** [October 7, 2019, 2:54pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599 "2019-10-07T14:54:32Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 7, 2019, 2:54pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/1 "2019-10-07T14:54:32Z")

</div>

I have a DataFrame df:

```julia
df = DataFrame(a = repeat([1, 2, 3, 4], outer=[2]),
                      b = repeat([2, 1], outer=[4]),
                      c = 1:8);

```

I want to apply the following operation:

```julia
operator=(:a => sum, :b => maximum)

```

to get the following result:

```julia
df_summary = DataFrame(a=sum(df.a), b=maximum(df.b))

```

My question: How can I apply the operator in a generic way to get the correct result? I cannot use the function “by”, because it fails when the group\_by clause is empty.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 7, 2019, 3:13pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/2 "2019-10-07T15:13:50Z")

</div>

I’m not sure I understand - you want to group reduce but the whole DataFrame is in the same group? Could you do

```julia
df = DataFrame(a = repeat([1, 2, 3, 4], outer=[2]),
                      b = repeat([2, 1], outer=[4]),
                      c = 1:8, d = 1)

by(df, :d, a_sum = :a => sum, b_max = :b => maximum)

```

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 7, 2019, 3:16pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/3 "2019-10-07T15:16:33Z")

</div>

> [@nilshg](#):
>
> by(df, :d, a\_sum = :a =\> sum, b\_max = :b =\> maximum)

This doesn’t work:

```julia
julia> by(df, :d, a_sum = :a => sum, b_max = :b => maximum)
ERROR: ArgumentError: column name :d not found in the data frame; existing most similar names are: :a, :b and :c
Stacktrace:

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 7, 2019, 3:17pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/4 "2019-10-07T15:17:39Z")

</div>

Sorry I should have been clearer in my post - I added `d = 1` to the call to the constructor of `df`, so you’d have to copy/paste my whole example!

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 7, 2019, 3:21pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/5 "2019-10-07T15:21:25Z")

</div>

Ok, this works, but it is not so nice. Firstly I need to extend the ordinal DataFrame with a column, choose a column name where I am sure it does not yet exist, than execute “by” and finally remove the redundant column.

Might cost some time and memory if the original dataframe has a million rows.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 7, 2019, 3:34pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/6 "2019-10-07T15:34:12Z")

</div>

Using now this code:

```julia
df = DataFrame(a = repeat([1, 2, 3, 4], outer=[2]),
                      b = repeat([2, 1], outer=[4]),
                      c = 1:8);

operator=(:a => sum, :b => maximum)

df_summary = DataFrame()
for pair in operator
    df_summary[!, pair[1]] = [pair[2](df[!, pair[1]])]
end

```

Seams to work, didn’t benchmark it yet.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 7, 2019, 3:39pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/7 "2019-10-07T15:39:49Z")

</div>

Yes something similar would have been my next suggestion - I wrote something quite similar albeit with slightly more complex column naming:

```julia
function dfreduce(df, f_pairs)
    r = DataFrame(x = f_pairs[1][2](df[!, f_pairs[1][1]]))
    names!(r, [Symbol(f_pairs[1][1], "_", f_pairs[1][2])])
    if length(f_pairs) > 1
        for e ∈ f_pairs[2:end]
            r[!, Symbol(e[1], "_", e[2])] .= e[2](df[!, e[1]])
        end
    end
    return r
end

function add_col_reduce(df, f_pairs)
    df[!, :for_grouping] .= 1
    by(df, :for_grouping, f_pairs)
end

using BenchmarkTools
@btime dfreduce(df, operator) # 10.499 μs (71 allocations: 5.06 KiB)
@btime add_col_reduce(df, operator) # 35.200 μs (183 allocations: 13.30 KiB) 

```

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 7, 2019, 3:52pm UTC](https://discourse.julialang.org/t/groupreduce-of-dataframe-with-empty-group/29599/8 "2019-10-07T15:52:37Z")

</div>

Thank you!  
I am nearly happy with my solution, but discussion with you helped!  
Only readability should be improved.

Nevertheless I think the “by” function should handle the case of an empty group\_by automatically. I might create an issue for that.