# How to compare non-missing elements of two DataFrames

**URL:** <https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326>\
**Category:** New to Julia\
**Created:** [June 30, 2020, 9:23pm UTC](https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326 "2020-06-30T21:23:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [June 30, 2020, 9:23pm UTC](https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326/1 "2020-06-30T21:23:42Z")

</div>

Seeking terseness in order to keep my @testset readable.  
In this particular @test, query responses (as DataFrames) need to be checked for accuracy.

There has to be a better way to do this:

```julia
response = DataFrame(:a=>["a",missing],:g=>["a",missing])
actual = DataFrame(:a=>["a",missing],:g=>["a",missing])
@test actual[:a][(!ismissing).(actual[:a])] == response[:a][(!ismissing).(actual[:a])] 
      && actual[:g][(!ismissing).(actual[:g])] == response[:g][(!ismissing).(actual[:g])]

```

I’m looking for something like the following, such that equality of non-missing values in all columns (:a, :g , etc) are checked without the columns themselves being listed explicitly

```julia
notmissing = !ismissing.(actual)
@test all(actual[notmissing] .== response[notmissing])

```

Edit: added multiple columns and corrected syntax on !ismissing

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [June 30, 2020, 11:55pm UTC](https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326/2 "2020-06-30T23:55:03Z")

</div>

You need `(!ismissing)` in parentheses.

---

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [July 1, 2020, 2:52am UTC](https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326/3 "2020-07-01T02:52:14Z")

</div>

Thanks for this, I have updated the original post for clarity

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [July 1, 2020, 3:04am UTC](https://discourse.julialang.org/t/how-to-compare-non-missing-elements-of-two-dataframes/42326/4 "2020-07-01T03:04:07Z")

</div>

For looking at the whole dataframe, could you do `dropmissing` for both data frames? I guess that wouldn’t allow for different columns to have different missing placement. You would be dropping too many rows.

Also, you should be using the two argument `getindex` method, instead of

```julia
actual[:a][(!ismissing).(actual[:a])]

```

do

```julia
actual[ismissing.(actual.a) .== false, :a]

```

But I think you should do

```julia
ulia> function check_nonmissing(df1, df2)
       for n in names(df)
           if collect(skipmissing(df1[!, n])) != collect(skipmissing(df2[!, n]))
               return false
           end
       end
       return true
       end

```