Skipping a whole `@testset`?

General Usage
4

Thanks @Satvik, but I was aiming in particular at the case when some tests are ‘broken’, rather than wanting to switch between them interactively.

I spent a few hours learning that wrapping @testset in another macro isn’t possible (at least for someone with my limited knowledge of macros), see Calling a macro from within a macro, revisited, https://github.com/JuliaLang/julia/issues/37691, and Nested macros and esc. After giving up on that, I ended up coming up with this macro that skips a testset and adds a DefaultTestSet with a dummy broken test so that it can show up in the summary:

import Test: Test, finish
using Test: DefaultTestSet, Broken
using Test: parse_testset_args

"""
Skip a testset

Use `@testset_skip` to replace `@testset` for some tests which should be skipped.

Usage
-----
Replace `@testset` with `@testset "reason"` where `"reason"` is a string saying why the
test should be skipped (which should come before the description string, if that is
present).
"""
macro testset_skip(args...)
    isempty(args) && error("No arguments to @testset_skip")
    length(args) < 2 && error("First argument to @testset_skip giving reason for "
                              * "skipping is required")

    skip_reason = args[1]

    desc, testsettype, options = parse_testset_args(args[2:end-1])

    ex = quote
        # record the reason for the skip in the description, and mark the tests as
        # broken, but don't run tests
        local ts = DefaultTestSet(string($desc, " - ", $skip_reason))
        push!(ts.results, Broken(:skipped, "skipped tests"))
        local ret = finish(ts)
        ret
    end

    return ex
end

I’d still vote for something like this as a feature of the standard-library @testset and DefaultTestSet.

6 Likes