# Delete missing values after the last non missing value in each id

**URL:** <https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [February 12, 2022, 9:29am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291 "2022-02-12T09:29:46Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![sai\_matcha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sai_matcha/32/35801_2.png) [@sai\_matcha](https://discourse.julialang.org/u/sai_matcha)\
**Post date:** [February 12, 2022, 9:29am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/1 "2022-02-12T09:29:46Z")

</div>

Hi , i have a dataframe which concatins missing values . i want to filter missing values after the last non missing value in the each id  
data frame i have is

```julia
df1 = DataFrame(id = [1,1,1,1,1,1] ,b = [1,2,missing,3,missing,missing])
df2 = DataFrame(id = [2,2,2,2,2,2] ,b = [1,2,missing,3,missing,5])
df3 = DataFrame(id = [3,3,3,3,3,3] ,b = [1,2,missing,3,4,missing])
df = [df1;df2;df3]

```

Here i have 2 missing values after “3” in df.b in id1 . similarly 1 in id 3. i wanted to filter/ remove those missing values.

DataFrame i want is

```julia
df1 = DataFrame(id = [1,1,1,1] ,b = [1,2,missing,3])
df2 = DataFrame(id = [2,2,2,2,2,2] ,b = [1,2,missing,3,missing,5])
df3 = DataFrame(id = [3,3,3,3,3,] ,b = [1,2,missing,3,4])
df = [df1;df2;df3]

```

Can someone help me to do this ?  
Thanks.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [February 12, 2022, 11:19am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/2 "2022-02-12T11:19:24Z")

</div>

Check this option:

```julia
vcat([g[1:findlast(!isequal(missing), g.b),:] for g in groupby(df,:id)]...)

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 12, 2022, 2:22pm UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/3 "2022-02-12T14:22:06Z")

</div>

With DataFramesMeta

```julia
julia> @chain df begin
           groupby(:id)
           @subset let
               last_missing_ind = findlast(ismissing, :b)
               1:length(:id) .<= last_missing_ind
           end
       end

```

---

<div class="post-metadata">

**Author:** ![sai\_matcha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sai_matcha/32/35801_2.png) [@sai\_matcha](https://discourse.julialang.org/u/sai_matcha)\
**Post date:** [September 1, 2022, 7:07am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/4 "2022-09-01T07:07:06Z")

</div>

Is there any update in this code. It worked few months back, now it gives below error

```julia
MethodError: no method matching (::Colon)(::Int64, ::Nothing)
Closest candidates are:
  (::Colon)(::T, ::Any, ::T) where T<:Real at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/range.jl:41
  (::Colon)(::A, ::Any, ::C) where {A<:Real, C<:Real} at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/range.jl:10
  (::Colon)(::T, ::Any, ::T) where T at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/range.jl:40
  ...
Stacktrace:
 [1] (::var"#65#66")(g::SubDataFrame{DataFrame, DataFrames.Index, Vector{Int64}})
   @ Main ./none:0
 [2] iterate
   @ ./generator.jl:47 [inlined]
 [3] collect_to!(dest::Vector{DataFrame}, itr::Base.Generator{GroupedDataFrame{DataFrame}, var"#65#66"}, offs::Int64, st::Int64)
   @ Base ./array.jl:782
 [4] collect_to_with_first!(dest::Vector{DataFrame}, v1::DataFrame, itr::Base.Generator{GroupedDataFrame{DataFrame}, var"#65#66"}, st::Int64)
   @ Base ./array.jl:760
 [5] collect(itr::Base.Generator{GroupedDataFrame{DataFrame}, var"#65#66"})
   @ Base ./array.jl:734
 [6] top-level scope
   @ none:1

```

---

<div class="post-metadata">

**Author:** ![sai\_matcha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sai_matcha/32/35801_2.png) [@sai\_matcha](https://discourse.julialang.org/u/sai_matcha)\
**Post date:** [September 1, 2022, 7:28am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/6 "2022-09-01T07:28:42Z")

</div>

This is the code i used

```julia
a = vcat([g[1:findlast(!isequal(missing), g.DV),:] for g in groupby(amik1a,:ID)]...)

```

Thanks

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [September 1, 2022, 9:12am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/7 "2022-09-01T09:12:26Z")

</div>

Your code is constructing a range:

```julia
1:findlast(!isequal(missing), g.DV)

```

where the endpoint is the result of `findlast`, which returns `nothing` if no element matches your predicate. This error:

```julia
MethodError: no method matching (::Colon)(::Int64, ::Nothing)

```

is telling you that there is no method for the operator `Colon` (which is `:`, the infix operator you’re using to construct your range) which takes an integer as first argument and `nothing` as a second.

While I can’t guarantee this without an MWE, I assume that `g.DV` is all missing values, so that `findlast` returns `nothing`, and you’re trying to construct the range `1:nothing` which doesn’t work:

```julia
julia> 1:findlast(!ismissing, [missing, missing, missing])
ERROR: MethodError: no method matching (::Colon)(::Int64, ::Nothing)

```

---

<div class="post-metadata">

**Author:** ![sai\_matcha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sai_matcha/32/35801_2.png) [@sai\_matcha](https://discourse.julialang.org/u/sai_matcha)\
**Post date:** [September 1, 2022, 9:54am UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/8 "2022-09-01T09:54:16Z")

</div>

Thank you very much

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [September 1, 2022, 9:51pm UTC](https://discourse.julialang.org/t/delete-missing-values-after-the-last-non-missing-value-in-each-id/76291/9 "2022-09-01T21:51:44Z")

</div>

```julia

function lostmissing(v)
  w=copy(v)
  while !isempty(w) && ismissing(last(w)) # it is important that! isempty (w) is checked before last (v)
    pop!(w)
  end
  return w
end

combine(groupby(df,:id), :b=>lostmissing)

```