# DataFrames: how to calculate conditional mean values in grouped dataframe?

**URL:** <https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194>\
**Category:** Statistics\
**Created:** [April 16, 2019, 8:45am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194 "2019-04-16T08:45:28Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [April 16, 2019, 8:45am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/1 "2019-04-16T08:45:28Z")

</div>

Dear Community,  
I have a DataFrame df, which I group for instance like:

```julia
foo = by(df, [:param1, :param2], :res => mean)

```

Here, columns param1 and param2 store parameter values and column res stores result values. Foo shows the mean result values for each parameter combination.

I also have another column :flag which stores exit flags (integers) of each result. How can the example above be adapted to consider only values in which certain exit flags are set? e.g., something like:

```julia
foo = by(df, [:param1, :param2], :res => mean if(:flag .= 1 || :flag.=2) )

```

Thank you in advance.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [April 16, 2019, 10:47am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/2 "2019-04-16T10:47:50Z")

</div>

On the phone so can’t check but I think

`by(df[df.flag.==1, :], [:param1, :param2], :res=>mean) `

Should do?

---

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [April 16, 2019, 11:01am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/3 "2019-04-16T11:01:59Z")

</div>

Thank you, this works!  
Unlikely I forgot to mention the following: suppose there is a parameter combination in which there is no df.flag .\<=2. These combinations are omitted in the resulting dataframe. Is there a way to force them to be listed (for instance with NaN values)?

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [April 16, 2019, 11:51am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/4 "2019-04-16T11:51:59Z")

</div>

> [@mike\_k](#):
>
> Unlikely I forgot to mention the following: suppose there is a parameter combination in which there is no df.flag .\<=2. These combinations are omitted in the resulting dataframe. Is there a way to force them to be listed (for instance with NaN values)?

Not currently. We could add an argument to enable that (e.g. `droplevels=false`), but note that in any case that would only work if `df.flag` is a `CategoricalArray`, since otherwise there’s no way `by` can know what are the possible values.

---

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [April 16, 2019, 11:57am UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/5 "2019-04-16T11:57:17Z")

</div>

Okay. Thank you anyway for your fast response 🙂

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [April 16, 2019, 2:15pm UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/6 "2019-04-16T14:15:59Z")

</div>

Create a new column that takes a value 1 when either `flag == 1` or `flag == 2` and 0 otherwise. Then also use this as a grouping column in the `by` command.

`by(df, [:param1, :param2, :newcol], :res=>mean)`

It will also compute the mean for values with `flag > 2`, you can just reset them to `NaN` if you want.

---

<div class="post-metadata">

**Author:** ![mike\_k](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike_k/32/211864_2.png) [@mike\_k](https://discourse.julialang.org/u/mike_k)\
**Post date:** [April 16, 2019, 2:23pm UTC](https://discourse.julialang.org/t/dataframes-how-to-calculate-conditional-mean-values-in-grouped-dataframe/23194/7 "2019-04-16T14:23:11Z")

</div>

This is a nice solution. Thank you!!