I have a few questions about unit testing workflows. Also, maybe we can use this thread to compile a list of best practices for unit testing which we can then contribute to the docs.
1 - where is
runtests.jl supposed to be run? When I run the tests, I load the environment of the project being tested. Also, sometimes I need to access files and folders relative to the current execution directory. I keep alternating between running from the root of the project and from the
test/ folder. Which is the right way?
P.S. - here it will be ideal to just be able to run
pkg> test as already discussed elsewhere.
2 - I’ve seen that there are test dependencies in
Project.toml. Is this feature documented and fully supported? I haven’t found the documentation although I’ve seen this used in the wild. Pkg doesn’t seem to support it out of the box - for instance, there’s no way to add an
extra. What’s the workflow? Add it as a dependency and move it by hand into
extras and add the test target? Will that be maintained by Pkg (updates, version bounds, etc).
3 - more towards CI, can anybody advise on testing against real databases (SQLite, MySQL, Postgres). Some sort of CI setup where you test without exposing the credentials – or some sort of DB testing services? Does anybody do this for open source projects?
In regards to best practices, I would like to mention
TestSetExtensions.jl (thanks @kristoffer.carlsson).
If you use
TestSetExtensions.jl for the usability, you can also break your tests into smaller files addressing one area of the tests and use the
@includetests macro to load them, ie:
@testset ExtendedTestSet "Genie tests" begin @includetests end
Currently broken on v1 but you can use my fork until support for Julia v1 is merged: