Unit testing with generated code

mike.ingold
Thu, 22 Feb 2024 23:52:00 +0000
General Usage

Problem Statement: I have a package where a number of functions are defined with multiple-dispatch signatures like f(x::A, y::B). Say that methods may be implemented for m types A and n types B and there are p functions, but not all methods will be defined. Currently two @tests are implemented for every supported combination. This leads to a support matrix where up to 2(m \times n \times p) tests should be defined. My goal is to implement something like a human-readable table where I can specify which methods are implemented and generate/run @tests for all of those.

Prior Attempts: My original solution was to manually write out all of the required tests, but this quickly became unwieldy. Eventually I consolidated things slightly using a for loop over types B. Recently I tried implementing the support table as a vector of named tuples and defined functions that implement the required tests for each named tuple, but it seems like trying to define @tests inside of a function, separate from the @testset is problematic. At a high level, it looked roughly of the form below:

function generatedtests(a,b)
    # ...
    @test ...
    @test ...
end

function test_row(nt)
    @testset “$typeof(nt.a)” begin
        foreach(generatedtest, ...)
    end
end

9 posts - 3 participants