# Split-Apply-Combine on many columns at once? Looking for equivalent to Stata's collapse

**URL:** <https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649>\
**Category:** New to Julia\
**Tags:** question, dataframes\
**Created:** [August 10, 2020, 12:47am UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649 "2020-08-10T00:47:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![clibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/clibassi/32/12856_2.png) [@clibassi](https://discourse.julialang.org/u/clibassi)\
**Post date:** [August 10, 2020, 12:47am UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/1 "2020-08-10T00:47:42Z")

</div>

Hi all,

I’ve got a dataset with multiple rows per individual that I would like to collapse to one row per individual, but I want to retain a variety of information for each individual (e.g. their sex, race, etc.) that are static across each observation and other things where I need to get the maximum value for each observation and others where I need the mean, etc. The upshot is that I need to collapse a bunch of different columns at once, performing different actions on each, some of which are numerical in nature, others categorical. I would normally do this in Stata with something like:

```julia
collapse (mean) test_score (max) wins_scholarship (first) gender race, by(id) 

```

Still trying to get the hang of split-apply-combine, so this may be obvious, but I would appreciate any help.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [August 10, 2020, 12:56am UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/2 "2020-08-10T00:56:28Z")

</div>

Its fairly similar. Here is the syntax for plain DataFrames.

```julia
combine(groupby(df,:id), :test_score => mean, :wins_scholarship => maximum, [:gender,:race] .=> first)

```

The names will be autogenerated like `:test_score_mean`. You can add `=> :newcolname` to any of those to manually name it.

---

<div class="post-metadata">

**Author:** ![clibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/clibassi/32/12856_2.png) [@clibassi](https://discourse.julialang.org/u/clibassi)\
**Post date:** [August 10, 2020, 1:11am UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/3 "2020-08-10T01:11:25Z")

</div>

Awesome - thank you!

---

<div class="post-metadata">

**Author:** ![clibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/clibassi/32/12856_2.png) [@clibassi](https://discourse.julialang.org/u/clibassi)\
**Post date:** [August 11, 2020, 2:39pm UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/4 "2020-08-11T14:39:29Z")

</div>

So I’m getting this error when I try your code on my data:

> MethodError: objects of type SubArray{Union{Missing, Float64},1,Array{Union{Missing, Float64},1},Tuple{Array{Int64,1}},false} are not callable  
> Use square brackets for indexing an Array.

I assume this is about the type of the underlying data? Another possibility is that it’s not possible to apply the same method to a group of columns in a square-brackets for traditional statistics, as you did with the first method in your example. Or maybe you have to be careful about grouping columns with different data types?

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [August 11, 2020, 3:04pm UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/5 "2020-08-11T15:04:06Z")

</div>

I’m guessing you might have missed the `.=>` (note the period).

This broadcasts `first` to each column:

```julia
[:gender,:race] .=> first

```

This tries to call `first(df.gender,df.race)`:

```julia
[:gender,:race] => first

```

---

<div class="post-metadata">

**Author:** ![clibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/clibassi/32/12856_2.png) [@clibassi](https://discourse.julialang.org/u/clibassi)\
**Post date:** [August 11, 2020, 4:20pm UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/6 "2020-08-11T16:20:21Z")

</div>

Oh yes - I’m dumb. That was it. Thanks so much!

---

<div class="post-metadata">

**Author:** ![jsaraviadrago](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jsaraviadrago/32/20358_2.png) [@jsaraviadrago](https://discourse.julialang.org/u/jsaraviadrago)\
**Post date:** [November 9, 2021, 10:52pm UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/7 "2021-11-09T22:52:20Z")

</div>

> [@tbeason](#):
>
> `using DataFrames, Pkg, CSV, Gadfly, HypothesisTests df_PS_resumen = combine(groupby(df_PS, [:Equipo]), :Goles => sum∘skipmissing, :Pases => maximum∘skipmissing, :Faltas => sum∘skipmissing)`

Hi everyone, I am also trying to aggregate and I am able to do it with sum and maximum but not with mean. I am pasting the code that works but If I change a parameter to for example :Faltas =\> mean∘skipmissing it says that mean isn’t found.

Thanks

---

<div class="post-metadata">

**Author:** ![jsaraviadrago](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jsaraviadrago/32/20358_2.png) [@jsaraviadrago](https://discourse.julialang.org/u/jsaraviadrago)\
**Post date:** [November 9, 2021, 11:52pm UTC](https://discourse.julialang.org/t/split-apply-combine-on-many-columns-at-once-looking-for-equivalent-to-statas-collapse/44649/8 "2021-11-09T23:52:30Z")

</div>

Found my answer: Using Statistics wasn’t loaded.