# Converting filter using "||" condition to subset

**URL:** <https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [May 15, 2021, 3:09pm UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203 "2021-05-15T15:09:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kobusherbst](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kobusherbst/32/19682_2.png) [@kobusherbst](https://discourse.julialang.org/u/kobusherbst)\
**Post date:** [May 15, 2021, 3:09pm UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203/1 "2021-05-15T15:09:38Z")

</div>

How do I convert the following code fragment to use `subset` rather than `filter` and is there a benefit in doing so for large dataframes?

```julia
df = DataFrame(a = [1,2,3,4,6,6], b=[1,missing,3,missing,5,missing], c=[missing,2,missing,4,missing,6])
filter([:a, :b] => (x, y) -> ismissing(y) || x > y, df)

```

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [May 15, 2021, 4:17pm UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203/2 "2021-05-15T16:17:39Z")

</div>

I don’t know if you accept this solution:

```julia
julia> function isgreater(x,y)
           if ismissing(x) || ismissing(y)
               return false
           end
           return x>y
       end

julia> subset(df, [:a,:b] => (x,y) -> ismissing.(y) .| isgreater.(x,y) )
4×3 DataFrame
 Row │ a b c
     │ Int64 Int64? Int64?
─────┼─────────────────────────
   1 │ 2 missing 2
   2 │ 4 missing 4
   3 │ 6 5 missing
   4 │ 6 missing 6

```

I don’t think `subset` is a benefit, a benchmark shows it’s slower.

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [May 15, 2021, 4:23pm UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203/3 "2021-05-15T16:23:19Z")

</div>

```julia
julia> @btime filter(:b => x -> ismissing(x), $df)
  1.330 μs (15 allocations: 1.42 KiB)

julia> @btime subset($df, :b => x -> ismissing.(x) )
  20.700 μs (146 allocations: 8.77 KiB)

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 15, 2021, 5:29pm UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203/4 "2021-05-15T17:29:13Z")

</div>

In this case `filter` is efficient and `subset` does more pre and post processing work, so it can be expected to be slower (but the slowdown should be roughly a constant - as your data grows the performance should be comparable).

---

<div class="post-metadata">

**Author:** ![qsong](https://avatars.discourse-cdn.com/v4/letter/q/d07c76/32.png) [@qsong](https://discourse.julialang.org/u/qsong)\
**Post date:** [May 16, 2021, 4:53am UTC](https://discourse.julialang.org/t/converting-filter-using-condition-to-subset/61203/5 "2021-05-16T04:53:49Z")

</div>

Why not just use `ByRow`?

```julia
subset(df, [:a, :b] => ByRow((x, y) -> ismissing(y) || x > y))

```