# Query.jl - filtering on missing data

**URL:** <https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898>\
**Category:** Data\
**Created:** [September 13, 2018, 8:40am UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898 "2018-09-13T08:40:24Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdsalerno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mdsalerno/32/5539_2.png) [@mdsalerno](https://discourse.julialang.org/u/mdsalerno)\
**Post date:** [September 13, 2018, 8:40am UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/1 "2018-09-13T08:40:24Z")

</div>

What is the correct way to filter on missing values? The following does not work properly:

```julia
result = @from r in df begin
    @where !ismissing(r.A)
    @select r
    @collect DataFrame
end

```

When I run such a query, the @where clause does not seem to recognize any values as missing. I encounter the same issue when using `!==` or `isequal()` in place of `ismissing()`. I did some searching and learned that there are compatibility issues between Query.jl and the Missing type, and that prior to Julia v0.7 `isnull()` produced the expected behavior however `isnull()` is no longer part of Base.

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [September 13, 2018, 11:41am UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/2 "2018-09-13T11:41:11Z")

</div>

I think you need to use `isna` now for DataValues; which Query.jl uses underneath for missing data.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [September 13, 2018, 5:31pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/3 "2018-09-13T17:31:36Z")

</div>

Yes, @quinnj is right, `isna` is the way to check for missing values in Queryverse/DataValues land on julia 0.7+.

---

<div class="post-metadata">

**Author:** ![mdsalerno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mdsalerno/32/5539_2.png) [@mdsalerno](https://discourse.julialang.org/u/mdsalerno)\
**Post date:** [September 13, 2018, 5:51pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/4 "2018-09-13T17:51:00Z")

</div>

That resolved it for me. Thanks @quinnj @davidanthoff.

In case anyone wonders, `isna` lives in DataValues.jl

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [September 13, 2018, 5:54pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/5 "2018-09-13T17:54:38Z")

</div>

Why isn’t it called `ismissing` as well for consistency? What can `na` be other then missing?

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [September 13, 2018, 5:58pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/6 "2018-09-13T17:58:18Z")

</div>

The semantics of `DataValue` and `Missing` are quite different, especially when it comes to predicates like `==`. I felt it was safer in that case to not reuse the `Missing` function names, but instead make it very clear that these are two different missing stories that behave differently.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [September 13, 2018, 6:00pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/7 "2018-09-13T18:00:47Z")

</div>

Makes sense, thanks.

---

<div class="post-metadata">

**Author:** ![jandehaan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jandehaan/32/6805_2.png) [@jandehaan](https://discourse.julialang.org/u/jandehaan)\
**Post date:** [September 21, 2018, 5:08pm UTC](https://discourse.julialang.org/t/query-jl-filtering-on-missing-data/14898/8 "2018-09-21T17:08:58Z")

</div>

Please clarify what those two different missing stories are or point to a resource that explains it. Thanks.