# ANN: Automated Property-Based Testing with Checkers.jl v0.0.1

**URL:** https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846
**Category:** Community
**Tags:** package, announcement, testing
**Created:** [July 14, 2017, 4:10am UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846 "2017-07-14T04:10:07Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![philip](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/philip/32/17191_2.png) [@philip](https://discourse.julialang.org/u/philip)
#### Post date: [July 14, 2017, 4:10am UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846/1 "2017-07-14T04:10:07Z")

</div>

Hi all, pleased to announce [https://github.com/pkalikman/Checkers.jl](https://github.com/pkalikman/Checkers.jl), a QuickCheck-like, property-based, automated randomized testing package for Julia. Please give it a try and let me know what you think.

```julia
julia> Pkg.add("Checkers")
INFO: Cloning cache of Checkers from https://github.com/pkalikman/Checkers.jl.git
INFO: Installing Checkers v0.0.1
INFO: Package database updated

julia> using Checkers

julia> f(x) = x^2
f (generic function with 1 method)

julia> @test_forall x in -10:10, f(x) >= 0 #Literally tests each value in the iterator
Test Passed
  Expression: (x in -10:10, f(x) >= 0)

julia> @test_formany -10<x<10, f(x) >= 0 #Randomly selects 100 values to test
Test Passed
  Expression: (:((-10 < x < 10, f(x) >= 0)), :(mode = test_formany))

julia> @test_forall x in -10:10, f(x) > 0 #Should fail b/c f(0) = 0
Test Failed
  Expression: (x in -10:10, f(x) > 0)
ERROR: There was an error during testing

julia> # Control how many tests in @test_formany:
       @test_formany ntests = 10_000 1 < x < 5, x^2 < 30
Test Passed
  Expression: (:(ntests = 10000), :((1 < x < 5, x ^ 2 < 30)), :(mode = test_formany))

julia> # Test a conditional property, passing only if 100 of the tests are not vacuous:
       @test_formany ntests = 100 0 < x < Inf, 0 < y < Inf, x < y --> log(x) < log(y)
Test Passed
  Expression: (:(ntests = 100), :((0 < x < Inf, 0 < y < Inf, $(Expr(:-->, :(x < y), :(log(x) < log(y)))))), :(mode = test_formany))

julia> # Test a conditional property, passing only if 100 of the tests are not vacuous, but only allow 100 tests. Should Error b/c not enough tests aren't vacuous:
       @test_formany ntests = 100 maxtests = 100 0 < x < 10, 0 < y < 10, x < y --> x^2 < y^2
Error During Test
  Test threw an exception of type ErrorException
  Expression:
(:(ntests = 100), :(maxtests = 100), :((0 < x < 10, 0 < y < 10, $(Expr(:-->, :(x < y), :(x ^ 2 < y ^ 2))))), :(mode = test_formany))
  Found only 50/100 values satisfying given condition.
  Stacktrace:
   [1] macro expansion at /Users/plk/.julia/v0.6/Checkers/src/./test-cases.jl:242 [inlined]
   [2] anonymous at ./<missing>:?
   [3] eval(::Module, ::Any) at ./boot.jl:235
   [4] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:66
   [5] macro expansion at ./REPL.jl:97 [inlined]
   [6] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
ERROR: There was an error during testing

julia> # Test a conditional property, passing only if 100 of the test are not vacuous, but allow 100,000 tests, so it should Pass:
       @test_formany ntests = 100 maxtests = 100_000 0 < x < 10, 0 < y < 10, x < y --> x^2 < y^2
Test Passed
  Expression: (:(ntests = 100), :(maxtests = 100000), :((0 < x < 10, 0 < y < 10, $(Expr(:-->, :(x < y), :(x ^ 2 < y ^ 2))))), :(mode = test_formany))

```

More details and examples at [Home - Checkers.jl](https://pkalikman.github.io/Checkers.jl/stable/)

Thanks!

---

<div class="post-metadata">

### Author: ![bicycle1885](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bicycle1885/32/107_2.png) [@bicycle1885](https://discourse.julialang.org/u/bicycle1885)
#### Post date: [July 14, 2017, 4:37am UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846/2 "2017-07-14T04:37:05Z")

</div>

Looks great!

Is that possible to specify a generator for `@test_formany` like  
`@test_formany x = randstring(10), length(x) == 10` ?

---

<div class="post-metadata">

### Author: ![Evizero](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evizero/32/10118_2.png) [@Evizero](https://discourse.julialang.org/u/Evizero)
#### Post date: [July 14, 2017, 11:01am UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846/3 "2017-07-14T11:01:04Z")

</div>

This looks interesting.

> [@philip](#):
>
> julia\> @test\_forall x in -10:10, f(x) \> 0 #Should fail b/c f(0) = 0  
> Test Failed  
> Expression: (x in -10:10, f(x) \> 0)  
> ERROR: There was an error during testing

From the perspective of someone who does use such kind of tests, the usability would increase a lot if the error message gave a sample value for which the tests failed

---

<div class="post-metadata">

### Author: ![philip](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/philip/32/17191_2.png) [@philip](https://discourse.julialang.org/u/philip)
#### Post date: [July 14, 2017, 12:49pm UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846/4 "2017-07-14T12:49:06Z")

</div>

This is a good idea. We have some custom generator support but it’s not particularly intuitive at the moment. Thanks!

---

<div class="post-metadata">

### Author: ![philip](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/philip/32/17191_2.png) [@philip](https://discourse.julialang.org/u/philip)
#### Post date: [July 14, 2017, 12:50pm UTC](https://discourse.julialang.org/t/ann-automated-property-based-testing-with-checkers-jl-v0-0-1/4846/5 "2017-07-14T12:50:09Z")

</div>

Also a great idea. We did think of this a little, and you can use the `logto` keyword with `@test_formany` to write the values tested to a `csv` and inspect, but indeed offering at least one failure case for each failure of each macro in the output is a great idea. Thanks!