# DataFrames unique - keep last occurence

**URL:** <https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723>\
**Category:** Data\
**Tags:** dataframes\
**Created:** [April 21, 2021, 10:31am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723 "2021-04-21T10:31:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Iulian.Cioarca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iulian.cioarca/32/30166_2.png) [@Iulian.Cioarca](https://discourse.julialang.org/u/Iulian.Cioarca)\
**Post date:** [April 21, 2021, 10:31am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/1 "2021-04-21T10:31:41Z")

</div>

I have the following dataframe, where :a and :b are some conditions and :c is an output:

```julia
 df = DataFrame(a=[1,1,2,2,1], b=[1,2,1,2,1],c=randn(5))

```

As you can see, the last row has the same conditions as first row (a=1, b=1). How can I filter this dataframe in order to remove duplicates based on columns :a and :b, but to keep the last occurence? The `unique` function keeps the first occurence. I tried to `reverse!` the dataframe and then apply `unique`, but this method is not implemented.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [April 21, 2021, 10:45am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/2 "2021-04-21T10:45:44Z")

</div>

you can do `sort(df, rev = true)` instead of `reverse`. see `? sort`. You can also do it in-place with `sort!`

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [April 21, 2021, 10:46am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/3 "2021-04-21T10:46:46Z")

</div>

or `combine(groupby(df, [:a, :b]), last)` (then instead of `last` you can choose any row selector so this is a more general approach)

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [April 21, 2021, 10:48am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/4 "2021-04-21T10:48:48Z")

</div>

rather than using `sort` I think it is better to write `df[reverse(axes(df, 1)), :]` in this approach (`df` in general might not be sortable)

---

<div class="post-metadata">

**Author:** ![Iulian.Cioarca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iulian.cioarca/32/30166_2.png) [@Iulian.Cioarca](https://discourse.julialang.org/u/Iulian.Cioarca)\
**Post date:** [April 21, 2021, 11:09am UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/5 "2021-04-21T11:09:14Z")

</div>

Thanks to both of you for the quick replies! This helps a lot! I had to do only one modification:  
`combine(groupby(df, [:a, :b]), last)`

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [April 21, 2021, 2:25pm UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/6 "2021-04-21T14:25:31Z")

</div>

Gents, sorry for my ignorance, but the solution issues error in Julia 1.6.0 Win10, DataFrames v0.21.8:

```julia
julia> combine(groupby(df, [:a, :b]), last)
ERROR: MethodError: no method matching combine(::GroupedDataFrame{DataFrame}, ::typeof(last))
Closest candidates are:
  combine(::AbstractDataFrame, ::Any...) at C:\Users\jrafa\.julia\packages\DataFrames\GtZ1l\src\abstractdataframe\selection.jl:537
  combine(::GroupedDataFrame; f...) at C:\Users\jrafa\.julia\packages\DataFrames\GtZ1l\src\groupeddataframe\splitapplycombine.jl:557
  combine(::Any, ::AbstractDataFrame) at C:\Users\jrafa\.julia\packages\DataFrames\GtZ1l\src\abstractdataframe\selection.jl:540
  ...
Stacktrace:
 [1] top-level scope
   @ REPL[5]:1

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [April 21, 2021, 2:33pm UTC](https://discourse.julialang.org/t/dataframes-unique-keep-last-occurence/59723/7 "2021-04-21T14:33:40Z")

</div>

you need to update you DataFrames.jl version.