# Unit test equality for arrays?

**URL:** <https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138>\
**Category:** General Usage\
**Created:** [January 3, 2018, 8:39pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138 "2018-01-03T20:39:19Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![fufjvnvnf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fufjvnvnf/32/2046_2.png) [@fufjvnvnf](https://discourse.julialang.org/u/fufjvnvnf)\
**Post date:** [January 3, 2018, 8:39pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/1 "2018-01-03T20:39:19Z")

</div>

I am trying to use unit test to test the equality of two arrays, wheres the following test:  
`@test [2.36248e-314, 2.36248e-314] ≈ [0.0, 0.0]`  
would fail. The two entries in the vector, if compared separately with the same method, would pass the equality test. Is there an extended unit test for equality for arrays in julia?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 3, 2018, 8:43pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/2 "2018-01-03T20:43:34Z")

</div>

> [@fufjvnvnf](#):
>
> The two entries in the vector, if compared separately with the same method, would pass the equality test.

What do you mean here?

```julia
julia> 2.36248e-314 ≈ 0.0
false

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [January 3, 2018, 8:46pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/3 "2018-01-03T20:46:12Z")

</div>

> [@fufjvnvnf](#):
>
> Is there an extended unit test for equality for arrays in julia?

There is (although as @kristoffer.carlsson pointed out, the two arguments are not \approx individually). You can see which method is being called for any function call in Julia with `@which`:

```julia
julia> @which [2.36248e-314, 2.36248e-314] ≈ [0.0, 0.0]
isapprox(x::AbstractArray, y::AbstractArray) in Base.LinAlg at linalg/generic.jl:1297

julia> @which 2e-314 ≈ 0.0
isapprox(x::Number, y::Number) in Base at floatfuncs.jl:205

```

and you can open the relevant line in your editor with `@edit`:

```julia
julia> @edit [2.36248e-314, 2.36248e-314] ≈ [0.0, 0.0]

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 3, 2018, 8:51pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/4 "2018-01-03T20:51:19Z")

</div>

To just test elementwise you could use dot syntax

```julia
julia> [1.0, 1e9] ≈ [0.0, 1e9]
true

julia> all([1.0, 1e9] .≈ [0.0, 1e9])
false

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 3, 2018, 9:14pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/5 "2018-01-03T21:14:57Z")

</div>

If you want to test that an array is approximately zero, you need to either pass an absolute tolerance (`atol` keyword) to `isapprox` (you can also do `@test x ≈ y atol=sometolerance` in 0.7) or just test `norm(x) ≤ sometolerance`.

Without an `atol` keyword, `≈` only tests that the _relative_ error is small, and in particular that about half of the significant digits match. `2.36248e-314 ≈ 0.0` is `false` because _none_ of the significant digits match.

The default `atol` is zero because there is no sensible way to pick a nonzero default absolute tolerance without additional information. The problem is that the absolute tolerance depends on the overall scale (it is “dimensionful”): the result of `x ≈ y` shouldn’t change if you change the units/scaling, e.g. if you multiply both `x` and `y` by `1e18`.

(I feel like this is a FAQ …)

---

<div class="post-metadata">

**Author:** ![fufjvnvnf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fufjvnvnf/32/2046_2.png) [@fufjvnvnf](https://discourse.julialang.org/u/fufjvnvnf)\
**Post date:** [January 3, 2018, 9:47pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/6 "2018-01-03T21:47:28Z")

</div>

I am actually surprised by this. I tested with 0.9999999999 and 1, which returns true, so I thought there is some default tolerance for equality testing.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 3, 2018, 9:50pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/7 "2018-01-03T21:50:07Z")

</div>

As Steven described, there exists a default relative tolerance.

Copying the docstring here:

```julia
 isapprox(x, y; rtol::Real=sqrt(eps), atol::Real=0, nans::Bool=false, norm::Function)

  Inexact equality comparison: true if norm(x-y) <= atol + rtol*max(norm(x), norm(y)). The default atol is zero and the default rtol depends on the types of x and y. The
  keyword argument nans determines whether or not NaN values are considered equal (defaults to false).

```

---

<div class="post-metadata">

**Author:** ![bramtayl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bramtayl/32/3614_2.png) [@bramtayl](https://discourse.julialang.org/u/bramtayl)\
**Post date:** [January 4, 2018, 1:20am UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/8 "2018-01-04T01:20:22Z")

</div>

Why didn’t `isapprox(x::AbstractArray, y::AbstractArray)` disappear as part of the . broadcasting deprecations?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 4, 2018, 1:45am UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/9 "2018-01-04T01:45:27Z")

</div>

Because it does something different from the dotted version.

Similar reason why `*(::Matrix, ::Matrix)` didn’t get removed.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [January 14, 2020, 11:52am UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/10 "2020-01-14T11:52:57Z")

</div>

I came across this thread after struggling to make sense out of tests for array elements. It might be worth explicating in the docs. It appears that `@test all(isapprox.(x, y, atol=0.05))` is the correct syntax (I couldn’t find anything simpler). One problem is `@test isapprox(x, y, atol=0.05)` passes with 2 elements per array and this could lead a person to construct the wrong tests in the more general case.

```julia
#Both pass
N = 2
x = fill(1.03, N)
y = fill(1.0, N)

@test all(isapprox.(x, y, atol=0.05))

@test isapprox(x, y, atol=0.05)

```

```julia
using Test
N = 5
x = fill(1.03, N)
y = fill(1.0, N)

# only the first passes
@test all(isapprox.(x, y, atol=0.05))

@test isapprox(x, y, atol=0.05)

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 14, 2020, 12:41pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/11 "2020-01-14T12:41:48Z")

</div>

As noted by Kristoffer Carlsson above,

```julia
isapprox(a, b)

```

compares under some norm determined by the arguments. It is different from

```julia
all(isapprox.(a, b))

```

It is really up to the user to select the best comparison — either one could be useful in some context.

It is important to note that a custom norm can be specified for `isapprox`, which allows things like

```julia
@test a ≈ b norm = my_custom_norm

```

in tests. I frequently find this useful.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 14, 2020, 12:58pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/12 "2020-01-14T12:58:49Z")

</div>

In particular, `isapprox(a, b, atol=1e-2)` is checking whether `norm(a - b) ≤ 1e-2`, where `norm` is the default Euclidean norm. It sounds like what you want (or _think_ you want) is the “[infinity norm](http://mathworld.wolfram.com/L-Infinity-Norm.html)”. If you define `norminf(x) = maximum(abs, x)` (or use `norm(x, Inf)` from the `LinearAlgebra` package), then you could employ the `norm=norminf` keyword as @Tamas_Papp suggested.

However, more generally I would tend to recommend _not_ using `atol` in floating-point approximate-equality tests — a more common choice should be `rtol` ( **relative** tolerance). For example,

```julia
@test x ≈ y rtol=0.05

```

(equivalently, `isapprox(x, y, rtol=0.05)`) tests whether `x` and `y` are within 5% of one another in the sense that `norm(x-y) ≤ rtol*max(norm(x), norm(y))`. The reason to use a relative tolerance is that it is **scale-invariant** (“dimensionless”): multiplying both `x` and `y` by any factor (changing the “units”) will not require you to change the test. (Note in particular that this test will always pass when `x = 1.03y` as in your example, for _any_ finite `x` and `y`.)

Moreover, all of the rules of floating-point arithmetic are designed to preserve _relative_ accuracy, not absolute accuracy. For example `x + y` in floating point is guaranteed to give the exact result up to a relative tolerance of the [machine precision](https://en.wikipedia.org/wiki/Machine_epsilon) `ε` (`eps(Float64) = 2.220446049250313e-16` in double precision), not including overflow. (If you perform multiple addition operations, however, these errors can accumulate.) So it is much easier to select a reasonable `rtol` than `atol`.

The `isapprox` function has a default `rtol` of `√ε`, which means that it checks whether about half of the significant digits match. This is a good default for most floating-point unit tests — coarse enough that it won’t give false negatives due to accumulated roundoff errors, but fine enough to catch most bugs that give the wrong answer (as opposed to bugs that simply exacerbate roundoff). Because of that, most floating-point unit tests can simply do `@test x ≈ y`.

---

<div class="post-metadata">

**Author:** ![tinosulzer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tinosulzer/32/23060_2.png) [@tinosulzer](https://discourse.julialang.org/u/tinosulzer)\
**Post date:** [April 21, 2021, 8:52pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/13 "2021-04-21T20:52:54Z")

</div>

I’ve found the

```julia
norminf(x) = maximum(abs, x)
@test x ≈ y rtol = 0.01 norm = norminf

```

approach to work well, and fail with a clearer error message than the

```julia
@test all(isapprox.(x, y, rtol = 0.01))

```

approach. Could it be made so that

```julia
@test x .≈ y rtol = 0.01

```

is equivalent to the norminf approach? This seems fairly intuitive to me. Currently the above line fails with error

```julia
Expression evaluated to non-Boolean
Expression: .≈(x, y, atol = 0.01)
Value: Bool[1, 1]

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 22, 2021, 2:41am UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/14 "2021-04-22T02:41:15Z")

</div>

> [@tinosulzer](#):
>
> ```julia
> @test x ≈ y rtol = 0.01 norm = norminf
> 
> ```
> 
> approach to work well, and fail with a clearer error message than the
> 
> ```julia
> @test all(isapprox.(x, y, rtol = 0.01))
> 
> ```

Note that these tests are _not_ mathematically equivalent.

Consider `x = [1,0]` and `y = [1,1e-6]`, in which case your first test passes while your second test fails. The first test is checking \Vert x - y \Vert\_\infty \le 0.01 \max \{ \Vert x \Vert\_\infty, \Vert y \Vert\_\infty \} (which is `true`), while the second test is checking |1 - 1| \le 0.01 \max \{ |1|, |1| \} (which is `true`) _and_ |0 - 10^{-6}| \le 0.01 \max \{ |0|, |10^{-6}| \} (which is `false`) _separately_.

> [@tinosulzer](#):
>
> Could it be made so that
> 
> ```julia
> @test x .≈ y rtol = 0.01
> 
> ```
> 
> is equivalent to the norminf approach?

This _could_ be done (because `@test` is a macro and can rewrite any valid syntax arbitrarily) but I don’t think it _should_ be done, because `.≈` suggests an elementwise `isapprox.(...)` call, which is _not_ the same as `isapprox` with an infinity norm as noted above.

---

<div class="post-metadata">

**Author:** ![tinosulzer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tinosulzer/32/23060_2.png) [@tinosulzer](https://discourse.julialang.org/u/tinosulzer)\
**Post date:** [April 22, 2021, 5:28pm UTC](https://discourse.julialang.org/t/unit-test-equality-for-arrays/8138/15 "2021-04-22T17:28:21Z")

</div>

Ok, makes sense