# Is there a package to compare if two DataFrames are the same?
**URL:** https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381
**Category:** New to Julia
**Created:** [August 6, 2020, 12:40am UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381 "2020-08-06T00:40:37Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [August 6, 2020, 12:40am UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/1 "2020-08-06T00:40:37Z")
Something that has more options than the `==` in Base for comparing dataframes? E.g. one where you can set the tolerance of difference in a float column. And also can compare columns by name even though they may be arranged in different orders?
Something like proc compare in SAS.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 6, 2020, 2:23pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/2 "2020-08-06T14:23:11Z")
Not aware of anything baked in, but seems easy enough to do using `isapprox`?
You’d just have to decide whether you want to decide equality elementwise, columnwise or across the whole DataFrame (i.e. if say you have an absolute tolerance of 0.1, do you consider two dfs equal if all elements are withing 0.1 of each other but in total the difference either in a column or across all columns is larger than 0.1?)
Just do something like:
```julia
julia> using DataFrames
julia> x = rand(10);
julia> x2 = copy(x); x2[end] += 0.01
0.7120634304599409
julia> df = DataFrame(a = x, b = x, c = rand(10));
julia> df2 = DataFrame(a = x, b = x2, c = rand(10));
julia> isapprox.(df, df2)
10×3 DataFrame
│ Row │ a │ b │ c │
│ │ Bool │ Bool │ Bool │
├─────┼──────┼──────┼──────┤
│ 1 │ 1 │ 1 │ 0 │
│ 2 │ 1 │ 1 │ 0 │
│ 3 │ 1 │ 1 │ 0 │
│ 4 │ 1 │ 1 │ 0 │
│ 5 │ 1 │ 1 │ 0 │
│ 6 │ 1 │ 1 │ 0 │
│ 7 │ 1 │ 1 │ 0 │
│ 8 │ 1 │ 1 │ 0 │
│ 9 │ 1 │ 1 │ 0 │
│ 10 │ 1 │ 0 │ 0 │
julia> isapprox.(df, df2, atol = 0.15)
10×3 DataFrame
│ Row │ a │ b │ c │
│ │ Bool │ Bool │ Bool │
├─────┼──────┼──────┼──────┤
│ 1 │ 1 │ 1 │ 0 │
│ 2 │ 1 │ 1 │ 0 │
│ 3 │ 1 │ 1 │ 1 │
│ 4 │ 1 │ 1 │ 0 │
│ 5 │ 1 │ 1 │ 0 │
│ 6 │ 1 │ 1 │ 0 │
│ 7 │ 1 │ 1 │ 0 │
│ 8 │ 1 │ 1 │ 0 │
│ 9 │ 1 │ 1 │ 0 │
│ 10 │ 1 │ 1 │ 1 │
```
and then use some combination of `any` and `all` depending on how you want to define whether they’re equal ot not.
---
### Author:  [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)
#### Post date: [January 15, 2024, 11:03am UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/3 "2024-01-15T11:03:18Z")
> [@nilshg](#):
>
> `isapprox.(df, df2, atol = 0.15)`
isapprox unfortunately fails if the column is for instance of DateTime type.
It would be really great if there were a function which knows that DataFrames can have multiple types, some of which need isapprox, some of which need isequal.
In best case, this would be part of DataFrames.jl
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 15, 2024, 11:46am UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/4 "2024-01-15T11:46:10Z")
I don’t see how this is related to DataFrames - if you are comparing collections of heterogeneous types, you need to decide what the right way to compare them is. You seem to want something like
```julia
mycompare(x::Number, y::Number) = isapprox(x, y)
mycompare(x, y) = isequal(x, y)
```
which is simple enough to define to suit your specific requirements.
---
### Author:  [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)
#### Post date: [January 15, 2024, 11:57am UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/5 "2024-01-15T11:57:46Z")
that is impressively simple, thank you for your help.
Still I think it would be a good addition to DataFrames.jl. There is more comparing two dataframes than comparing its values - also the columns needs to be compared.
But I admit, this little helper is very compact and self-explaining.
---
### Author:  [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [January 15, 2024, 12:30pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/6 "2024-01-15T12:30:33Z")
DataFrames already has a definition of `isapprox` that does the right thing comparing columns by pairs:
>
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 15, 2024, 1:29pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/7 "2024-01-15T13:29:38Z")
Ha, that function seems to have been added just when I proposed the `isapprox` solution above.
But this also doesn’t help Stephan, who wants non-numerical types to be compared by `isequal`, which this definition also doesn’t do:
```julia
julia> df1 = DataFrame(a = [1,2], b = ["a", "b"]); df2 = copy(df1);
julia> isapprox(df1, df2)
ERROR: MethodError: no method matching -(::String, ::String)
```
---
### Author:  [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [January 15, 2024, 1:53pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/8 "2024-01-15T13:53:58Z")
Indeed, I was just replying to this part:
> [@schlichtanders](#):
>
> I think it would be a good addition to [DataFrames.jl](https://juliahub.com/ui/Packages/DataFrames). There is more comparing two dataframes than comparing its values - also the columns needs to be compared.
The scalar comparison issue remains but as you said above, that’s not a DataFrames issue.
---
### Author:  [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)
#### Post date: [January 15, 2024, 2:24pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/9 "2024-01-15T14:24:26Z")
Cool.
For me it is a DataFrame issue. DataFrames have different types of columns, by nature. Hence if someone shows me a method which is intended to work generically on DataFrames, I expect it to work on DataFrames with different kinds of columntypes.
The `isapprox` on DataFrames is hence a bit odd to me, as it only works for DataFrames of Numbers, but the signature really suggests that it works for all kinds of DataFrames.
---
### Author:  [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [January 15, 2024, 2:46pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/10 "2024-01-15T14:46:07Z")
The documentation says “`isapprox` with given keyword arguments applied to all pairs of columns” which implies that this method must be defined for all pairs of column types…
I agree it could be made more explicit in the documentation. But this restriction makes sense:
- the column types can be anything, there’s no way DataFrames.jl can know them all.
- DataFrames.jl is not the right place for defining `isapprox` on `DateTime` or other non-DataFrames types. Actually defining these methods in DataFrames.jl would be type piracy!
---
### Author:  [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)
#### Post date: [January 15, 2024, 2:59pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/11 "2024-01-15T14:59:56Z")
Yeah there is some danger of type-piracy in the air.
two ways I see how DataFrames isapprox could be implemented to still support different column types:
1. While restricting `isapprox` to only `Number` columns won’t work apparently, one could make this a configuration (which may even default to `Number`, but can be set to any type, including of course `Union` types)
2. You can have a try-catch fallback in case `isapprox` is not defined. That is probably slow, admitted, but would solve this.
In end-user code which is not a package used anywhere, you could probably also safely define the isapprox fallback yourself:
```julia
isapprox(a, b; kargs...) = isequal(a, b)
```
which at least semantically makes sense, as if something isequal, then it is also approximately equal.
But even though this is not type piracy, it is probably not a good idea to implement it in a package because people might be confused if no error is thrown for their non-number types.
---
### Author:  [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)
#### Post date: [January 15, 2024, 3:16pm UTC](https://discourse.julialang.org/t/is-there-a-package-to-compare-if-two-dataframes-are-the-same/44381/12 "2024-01-15T15:16:05Z")
I’m not sure it has the functionality you seek (didn’t look into it), but this package announcement seems to be aligned with your goal of better Data Frame comparisons. May want to add new features/requests there:
> [@\[ANN\] TestingUtilities.jl](https://discourse.julialang.org/t/ann-testingutilities-jl/99121/5#dataframes-julia-19-only-3):
>
> Version 1.3.0 Version 1.3.0 is out now and brings some convenient features for failing equality tests, e.g., @Test x == y or @Test isequal(x,y). When x and y have specific types listed below, TestingUtilities will print out a nicely formatted message about why specifically these two quantities are not equal (similar in spirit to [WhyNotEqual.jl](https://github.com/jw3126/WhyNotEqual.jl)). Strings If x and y are two Strings, the failing test will display the common prefix of x and y in green and the subsequently differing suffix in red, e…