# Dataframes subset w/ nullable logic

**URL:** <https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484>\
**Category:** General Usage\
**Tags:** dataframes, boolean\
**Created:** [July 25, 2024, 5:02pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484 "2024-07-25T17:02:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mikemartind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikemartind/32/210913_2.png) [@mikemartind](https://discourse.julialang.org/u/mikemartind)\
**Post date:** [July 25, 2024, 5:02pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484/1 "2024-07-25T17:02:21Z")

</div>

Working on a subset call that filters on multiple columns in a dataframe, but with variable conditions set by an external dict. So if the dict has a value, it checks for equality in the corresponding column. If the dict has a “null”, in this case, an empty string, it allows all rows (but lets the other columns filter in their own right). This means that the call looks like:  
`subset(data, :X => x -> (x.=="") .|| (x.==criteria['X']), :Y => y-> (y.=="") .|| (y.==criteria["Y"]))`  
What am I doing wrong w/ this logic? The short-circuiting should allow a bunch more rows through, but instead I’m getting all rows filtered out.

---

<div class="post-metadata">

**Author:** ![chris-b1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chris-b1/32/14165_2.png) [@chris-b1](https://discourse.julialang.org/u/chris-b1)\
**Post date:** [July 25, 2024, 6:43pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484/2 "2024-07-25T18:43:51Z")

</div>

There’s nothing obviously wrong there, try getting your problem down to a fully reproducible example with data. E.g. below works fine

```julia
julia> df = DataFrame(X=["", "A", "B"], Y=["A", "", "B"])
3×2 DataFrame
 Row │ X Y
     │ String String
─────┼────────────────
   1 │ A
   2 │ A
   3 │ B B

julia> criteria = Dict("X" => "A", "Y" => "A")
Dict{String, String} with 2 entries:
  "Y" => "A"
  "X" => "A"

julia> subset(df, :X => x -> (x.=="") .|| (x.==criteria["X"]), :Y => y-> (y.=="") .|| (y.==criteria["Y"]))
2×2 DataFrame
 Row │ X Y
     │ String String
─────┼────────────────
   1 │ A
   2 │ A

```

---

<div class="post-metadata">

**Author:** ![mikemartind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikemartind/32/210913_2.png) [@mikemartind](https://discourse.julialang.org/u/mikemartind)\
**Post date:** [July 25, 2024, 7:41pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484/3 "2024-07-25T19:41:13Z")

</div>

I should clarify, the dataframe doesn’t have “” in it, the criteria might, so if there are any “” criteria, I want that arg to be True all the time.

---

<div class="post-metadata">

**Author:** ![chris-b1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chris-b1/32/14165_2.png) [@chris-b1](https://discourse.julialang.org/u/chris-b1)\
**Post date:** [July 25, 2024, 8:14pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484/4 "2024-07-25T20:14:14Z")

</div>

In that case you’d need to make the conditional on the criteria - like this.

```julia
subset(df, :X => x -> criteria["X"] == "" ? trues(length(x)) : (x .== criteria["X"]),
           :Y => y -> criteria["Y"] == "" ? trues(length(y)) : (y .== criteria["Y"]))

```

Though, especially if the number of criteria is large or dynamic, I might write it more like this

```julia
julia> function make_subset(criteria)
           subsets = []
           for (name, value) in criteria
               if value != ""
                   push!(subsets, name => (x -> x .== value))
               end
           end
           return subsets
       end
make_subset (generic function with 1 method)

julia> subset(df, make_subset(criteria)...)
1×2 DataFrame
 Row │ X Y
     │ String String
─────┼────────────────
   1 │ C A

```

---

<div class="post-metadata">

**Author:** ![mikemartind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikemartind/32/210913_2.png) [@mikemartind](https://discourse.julialang.org/u/mikemartind)\
**Post date:** [July 25, 2024, 9:07pm UTC](https://discourse.julialang.org/t/dataframes-subset-w-nullable-logic/117484/5 "2024-07-25T21:07:36Z")

</div>

Realized it myself while I was running errands. Thanks for the fresh eyeballs!!!