Here is a conundrum I have about unit testing for our brilliantly conceived Julia packages (and unit testing in general)
Let us assume that I have invented an amazing algorithm to compute the meaning of life. As a good Julian, I write a package MeaningOfLife.jl to implement my amazing algorithm. Let us further assume that both the algorithm and my Julia implementation thereof are minimally complex. As a good citizen, I begin to write some unit tests.
Here’s the rub: In order to verify that the call
meaning_of_life(l::Life) produces correct results, I have to implement the algorithm in the test module. But by assumption, the algorithm is minimally complex. By further assumption, my Julia implementation to be tested is a minimally complex implementation thereof. Therefore because the tests are also written in Julia, all I am really doing in the unit test module is writing the same code over again, with at least as much complexity in the second implementation. If bugs follow a Poisson process, then my unit test is at least as likely to be wrong as what it is testing!
Thus, we see that unit tests are apparently useless. This brings up a further paradox: It does not seem reasonable that unit tests are useless, given that they are so widely used. What am I missing?