Like many users of Julia, I’m an engineer who mostly uses it to write one-off
scripts and do ad-hoc calculations. But I’ve also started to try to wrap up
some of my more commonly-used functionality into packages for re-use and finding
out: I have no idea how to write proper tests.
The two big questions I can’t seem to figure out are:
How big of pieces of functionality count as a unit and should have tests
written for them?
How do you structure a package so that it’s easily testable? I find this
especially difficult since some of the packages I’m writing wrap
functionality from other sources, and I don’t really want to test that the
external dependency is working as expected nor require installation of the
external dependency for execution of tests.
Any advice? Or maybe a good tutorial on testing that’s language-agnostic enough
to be applicable to Julia?
You learn these by doing, it is difficult to give general advice. There are books on the topic, but the best teacher is experience, along the lines of “if I tested that I would have saved two days of debugging”. That said,
you can (and should) test small pieces when it makes sense,
break up things into small functions, which is good style anyway, and the compiler likes it too, so it is an overall win.
you don’t need to test functions from other packages. If you find something lacking coverage, just make a PR for that package.
To be more clear, I’m specifically talking about wrapping non-Julia stuff in
point 2. For example, this weekend I hacked together a lightweight wrapper for taskwarrior. Most of the top-level functions
eventually call a function that calls taskwarrior. How do I structure the
library so that I can test that kind of stuff without actually calling the task binary?
Hard to say without seeing your code. If, for example, you have functions that emit/parse data for/from an external library, you can test these.
Note, however, that your will not be testing important functionality if you don’t test the actual interface to this library. Setting up a testing framework for this is usually more involved, eg your test script would need to download and install it.
I don’t think you can really test a wrapper, without having the external dependency available for the tests.
If there is some stuff that you want to use in the tests, which isn’t required for people just using the package, then you can have a separate REQUIRE file in your test subdirectory.
For 2. I think you want to test that your wrapper interact correctly with the library, say if you have a launch function that starts taskwarrior, you want to check that it actually starts it. When it comes to dependencies, most packages download/install/build them for you when it’s possible, and that’s part of the tests (it’s also very tedious to test it manually on different machines, so having it in the tests saves you some pain). Since taskwarrior seems to be available on most package manager that shouldn’t be too hard to do with BinDeps.
Thanks everyone. This was all helpful as basic as I’m sure it seemed to y’all. I
think part of the problem I was having was that I was laboring under the
impression that the units in “unit testing” were separate pieces of code that
should only be tested independently from each other. There was a comment made on
gitter today saying that when testing, you should think of units building up
into bigger units building up into bigger units, and tests should be done at
each level of agglomeration. Thinking about it that way, the proper approach
seems much more clear.
Anyway, thanks again for entertaining the super-basic questions of an engineer
who doesn’t really know what he’s doing in software.
