Getting started with unit testing in Julia

#1

I’m coming from a (mostly) Fortran/Python background.

Can someone give me a place to start for writing/running unit tests for my Julia code? A tutorial somewhere perhaps? And getting coverage testing and CI running? I’ve written a bit in Julia already but I’m ready to get serious and I’d like to maintain the good software engineering habits I tried to foster when I did a lot of python…

#2

The standard library Test is great for testing things. It provides a @test macro which can be used for most things, and a few other macros for special cases. There’s also @testset to group your tests, though that’s not mandatory. A taste:

julia> using Test

julia> @test 1 == 1
Test Passed

julia> @test 1 == sum([1,2])
Test Failed at REPL[3]:1
  Expression: 1 == sum([1, 2])
   Evaluated: 1 == 3
ERROR: There was an error during testing

julia> @testset "some tests" begin
           # Simple tests which pass
           @test 1 == 1
           @test sum([1,2,3]) == 6
           
           # Numerical testing with `isapprox` (\approx ≈):
           @test sqrt(2) ≈ 1.414213562
           # The following works to supply tolerance keywords to ≈:
           @test sqrt(2) ≈ 1.4142135623730951   rtol=1e-16 atol=0

           # Tests which fail give nice error reporting
           a = [1,2,3]
           @test sum(a) == 7
           
           # Exceptional circumstances can be tested for
           @test_throws DivideError 1÷0
           
           # Tests which currently don't work can be documented as such
           @test_broken 1 == 2
           # Tests which are so broken they shouldn't be run can be skipped
           @test_skip just_broken
       end

some tests: Test Failed at REPL[10]:13
  Expression: sum(a) == 7
   Evaluated: 6 == 7
Stacktrace:
 [1] macro expansion at ./REPL[10]:13 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [3] top-level scope at ./REPL[10]:3
Test Summary: | Pass  Fail  Broken  Total
some tests    |    5     1       2      8
ERROR: Some tests did not pass: 5 passed, 1 failed, 0 errored, 2 broken.

To use Test most effectively, you should put your code into a proper package structure which will allow you to use Pkg.test (or simply test YourPkg from the pkg REPL mode). I suggest using PkgTemplates.jl to generate a standard package structure including your choice of CI config for Travis / Appveyor / GitLab (/ others?), automatic upload of coverage data to Coveralls or CodeCov and other package stuff like the package definition, license file etc.

#3

I used PkgTemplates to create a package. It creates a file ~/src/foo/test/runtests.jl , which contains a test that 1 == 2, which of course fails. When you have some code to test, you put the test there instead of “@test 1 == 2”.

#4

Thanks for the input.

A 2 minute walkthrough to set up a package using PkgTemplates would be nice…

#5

This is open source. Now when you have figured that out, you can make the video / blog post and post the link here.