# Easier way to split-apply-combine in DataFrames.jl

**URL:** <https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [December 13, 2020, 10:35pm UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779 "2020-12-13T22:35:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![stefanjwojcik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefanjwojcik/32/14626_2.png) [@stefanjwojcik](https://discourse.julialang.org/u/stefanjwojcik)\
**Post date:** [December 13, 2020, 10:35pm UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/1 "2020-12-13T22:35:36Z")

</div>

I am updating some code that uses DataFrames.jl. I formerly used the `by` command and the `aggregate` command to execute commands over multiple columns, but it looks like these commands were deprecated recently. The updates for these functions seem a bit clunky.

If I have a dataframe like this:

```julia
using DataFrames
using Statistics
julia> df = DataFrame(x=rand(5), z=rand(5), y=["this", "that", "this", "that", "this"])
5×3 DataFrame
 Row │ x z y      
     │ Float64 Float64 String 
─────┼────────────────────────────
   1 │ 0.1203 0.562386 this
   2 │ 0.828021 0.355487 that
   3 │ 0.218044 0.859215 this
   4 │ 0.500774 0.329637 that
   5 │ 0.95807 0.39004 this

```

If I want to group by y and get the means, I used to do this:

```julia
julia> aggregate(df, :y, mean)

2×3 DataFrame
 Row │ y x_mean z_mean   
     │ String Float64 Float64  
─────┼────────────────────────────
   1 │ this 0.289291 0.432779
   2 │ that 0.886162 0.66099

```

Now it looks like the canonical way to do it is this:

```julia
julia> combine(groupby(df, :y), [:x, :z] .=> mean)
2×3 DataFrame
 Row │ y x_mean z_mean   
     │ String Float64 Float64  
─────┼────────────────────────────
   1 │ this 0.289291 0.432779
   2 │ that 0.886162 0.66099

```

Did I miss something or is this the most code-efficient way of achieving this?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 13, 2020, 10:36pm UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/2 "2020-12-13T22:36:16Z")

</div>

That’s the correct way!

---

<div class="post-metadata">

**Author:** ![stefanjwojcik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefanjwojcik/32/14626_2.png) [@stefanjwojcik](https://discourse.julialang.org/u/stefanjwojcik)\
**Post date:** [December 13, 2020, 10:38pm UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/3 "2020-12-13T22:38:09Z")

</div>

I can see some advantages when applying different functions to different subsets of columns, but it seems like two functions is replacing one for quick aggregations. Perhaps I’m missing some deeper reasoning here.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 14, 2020, 12:53am UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/4 "2020-12-14T00:53:23Z")

</div>

I think the deeper reasoning is that the scenario where all your columns are `<:Real` is a bit of an edge case. It’s not worth maintaining `aggregate` just for that scenario when `combine` works just as well.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [December 14, 2020, 2:43am UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/5 "2020-12-14T02:43:47Z")

</div>

> [@stefanjwojcik](#):
>
> Perhaps I’m missing some deeper reasoning here.

I wasn’t involved in the decision, and I initially thought it was a bit clunky too. But the promise of this approach is a powerful, concise, and consistent syntax for all kinds of transformations. Fwiw, after using it for a bit I think this promise has been fulfilled.

Now the API is unified for v1.0. I suspect that some conveniences will be added back for simple stuff like this, but even if not, I think the benefits of the new system are well worth it.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [December 14, 2020, 10:03pm UTC](https://discourse.julialang.org/t/easier-way-to-split-apply-combine-in-dataframes-jl/51779/6 "2020-12-14T22:03:01Z")

</div>

Yes, the idea is that it’s better to have a single function to perform the `combine` step, which is general enough to cover all use cases. Having both `combine` and `aggregate` makes the API more difficult to master, especially since the two names are very similar and nothing indicates which does what. By following this principle everywhere, we get a package which is much easier to use.

Also, following what @pdeffebach said, if your data suddenly gets a second string column, `combine(groupby(df, :y), [:x, :z] .=> mean)` will continue to work, contrary to `aggregate`. Things like `combine(groupby(df, :y), names(df, Real) .=> mean)` can also be used to compute the mean of all numeric columns.