RandomizedPropertyTest.jl is a test framework for testing program properties with random (as well as special pre-defined) inputs, inspired by QuickCheck . Sources can be found at Lukas Himbert / RandomizedPropertyTest.jl · GitLab and ~quf/RandomizedPropertyTest.jl - Randomized property tests in Julia. Inspired by https://github.com/nick8325/quickcheck. - sourcehut git. Documentation is also found there. It works for any Julia version from 1.0¹. The current version is 0.1.0, so the API is not guaranteed to be stable. However, I think the current API works quite well and I don’t plan on changing it without good reason.
Example use:
julia> using RandomizedPropertyTest
julia> @quickcheck (x^2 ≥ 0) (x :: Float64)
┌ Warning: Property `x ^ 2 ≥ 0` does not hold for x = NaN.
└ @ RandomizedPropertyTest [snip]/RandomizedPropertyTest.jl:[snip]
false
julia> @quickcheck (sin(x + π/2) ≈ cos(x)) (x :: Range{Float64, 0, 2π})
true
julia> @quickcheck n=10^6 (sum(x^k/factorial(k) for k in 20:-1:0) ≈ exp(x)) (x :: Range{Float64, -2, 2})
true
It has builtin support for a few basic Julia datatypes and can be extended for custom datatypes or custom random distributions (see README.md).
I’d be happy to receive any feedback, bug reports, suggestions, &c.
I would especially like some suggestions on how to deal with combinatorial explosion of special cases, as well as parallelism:
Special cases are defined for each basic data type (e.g. NaN for AbstractFloat). For multiple data types (or tuples), all combinations of special cases are checked. This leads to an exponential increase in special cases to be checked. Of course, I could check only a subset of special cases, but it’s not clear to me what that subset should be.
Parallelism would obviously be beneficial for performance, but I would like to keep deterministic test results to aid debuggability. In sequential execution it is achieved by using a PRNG with fixed seed. In a parallel setting multiple PRNGs with different seeds would need to run in parallel depending on the number of threads. Re-running the same check with a different number of threads may give a different result in that case.
¹ Unit tests for 0.1.0 (not master) are broken on Julia 1.0, but this only affects the package’s tests, not the package.