I am wondering about how to organize test code that gets re-used, but not in a pattern that is regular for all cases. Eg consider the MWE
using Test
function dotests(xs)
for x in xs
@test isodd(x)
end
end
function dotests2(xs)
for x in xs
@test x ≤ 7
end
end
@testset "some numbers" begin
xs = 1:2:7
dotests(xs)
dotests2(xs)
end
@testset "some other numbers" begin
dotests(3:4:17)
# NO dotests2 here
end
which seems to work, but relies on the concept of the “active test” (cf Test.get_testset()) to make sure results end up in the right place. The latter is undocumented so I am only somewhat certain it is part of the API.
I have seen some packages do giant loops, but that works less well when not every component applies to all cases.
One way to avoid this would be to have your dotests functions not to do the tests but to return the Booleans, and then test that locally within your testset.
That would aggregate test information for no good reason. Also, I am not sure there is anything here to avoid. I am quite convinced that this is OK, just underdocumented.
Yea… I thought you’d object to that. My line of thought is that tests are not meant to be quick (cause everything else takes a lot longer than the actual test), so unnecessary aggregation of data is not the end of the world.
Cool cool cool cool. I’m curious to learn the answer as well.
If I remember correctly this has the drawback that failing tests will report the line number of the function definition rather than the line number of the call.
So you might get confused whether it failed in “some numbers” or “some other numbers” (in this case it is actually clear, but if the testsuite is has more than two invocations, it is more of a problem).
That is why I usually define macros instead of functions to pass the line number information.