# How can I skip missing values of a DF without deleating them?

**URL:** <https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [November 11, 2021, 10:42pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354 "2021-11-11T22:42:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![SamanthaCassab](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samanthacassab/32/30672_2.png) [@SamanthaCassab](https://discourse.julialang.org/u/SamanthaCassab)\
**Post date:** [November 11, 2021, 10:42pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354/1 "2021-11-11T22:42:53Z")

</div>

Hi, I have a DataFrame with NaN and missing values. And I´m trying to use the function groupby and combine, with a function I define as:

```julia
flujos(x)=mean(filter(!isnan,skipmissing(x - lag(x)))) 

```

And then I use gpb and combine as:

```julia
df = DataFrame(A = [1:2;4;4;4], B = [5, 1, 2, NaN,8], C=[5:8;3], D=[9:12;7], E=[13:16;6]);
gdf=groupby(df,:A)
combine(gdf, :C =>flujos)

```

But I have a TypeError that says " reducing over an empty collection is not allowed" wich I asume is because my function deletes rows from that column and obviously a DataFrame can´t work with it, but I don´t know how to skip the missing values and do operations over it. Thank u.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 11, 2021, 10:47pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354/2 "2021-11-11T22:47:49Z")

</div>

I get a different error than you

```julia
julia> using ShiftedArrays, Statistics, DataFrames;

julia> flujos(x)=mean(filter(!isnan,skipmissing(x - lag(x))))
flujos (generic function with 1 method)

julia> df = DataFrame(A = [1:2;4;4;4], B = [5, 1, 2, NaN,8], C=[5:8;3], D=[9:12;7], E=[13:16;6]);

julia> gdf=groupby(df,:A)
GroupedDataFrame with 3 groups based on key: A
First Group (1 row): A = 1
 Row │ A B C D E     
     │ Int64 Float64 Int64 Int64 Int64 
─────┼─────────────────────────────────────
   1 │ 1 5.0 5 9 13
⋮
Last Group (3 rows): A = 4
 Row │ A B C D E     
     │ Int64 Float64 Int64 Int64 Int64 
─────┼─────────────────────────────────────
   1 │ 4 2.0 7 11 15
   2 │ 4 NaN 8 12 16
   3 │ 4 8.0 3 7 6

julia> combine(gdf, :C =>flujos)
ERROR: ArgumentError: reducing over an empty collection is not allowed

```

---

<div class="post-metadata">

**Author:** ![SamanthaCassab](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samanthacassab/32/30672_2.png) [@SamanthaCassab](https://discourse.julialang.org/u/SamanthaCassab)\
**Post date:** [November 11, 2021, 10:49pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354/3 "2021-11-11T22:49:26Z")

</div>

Oh yes that´s my error y copied the wrong error. (ups)

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 11, 2021, 10:52pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354/4 "2021-11-11T22:52:09Z")

</div>

The error should be self explanatory. The first group has one observation, so `x - lag(x)` is just `missing`, so the mean of `skipmissing(x - lag(x))` undefined because `skipmissing(x - lag(x))` is empty.

You should handle this case explicitely in your `flujos` function.

---

<div class="post-metadata">

**Author:** ![SamanthaCassab](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samanthacassab/32/30672_2.png) [@SamanthaCassab](https://discourse.julialang.org/u/SamanthaCassab)\
**Post date:** [November 11, 2021, 10:59pm UTC](https://discourse.julialang.org/t/how-can-i-skip-missing-values-of-a-df-without-deleating-them/71354/5 "2021-11-11T22:59:46Z")

</div>

Ok, but I was looking for a function that skips missing values without deleating them, if there´s so. If not I guess I ´ll have to handle it in my flujos function.  
Thank u.