Incremental testing

General Usage
1

Hi,

I was reading the page on unit testing and I found that the suggested method ] test was rather slow due to pre-compilation, making it unusable for me during development.

There is a much faster way to incrementally run tests, using Revise.jl.

For example I have a file test/runtests.jl (from BestieTemplate.jl) which collects my tests.

#=
Don't add your tests to runtests.jl. Instead, create files named

    test-title-for-my-test.jl

The file will be automatically included inside a `@testset` with title "Title For My Test".
=#
function runtests()
    for (root, dirs, files) in walkdir(@__DIR__)
        for file in files
            if isnothing(match(r"^test-.*\.jl$", file))
                continue
            end
            title = titlecase(replace(splitext(file[6:end])[1], "-" => " "))
            @testset "$title" begin
                @info "Running test: $file at dir $root"
                include(root * '/' * file)
            end
        end
    end
end

# Run the tests
runtests()

From Revise.jl I call includet to track any changes to my tests:

julia> using Revise
includet("test/runtests.jl")
runtests()

Now I can run my tests, make changes, create new tests all without having to pre-compile.

Would it make sense to add this as a section to the unit testing docs?

1 Like
2

I think it would be great to add something similar, or post a library that makes unit tests with Revise easier.

I have something similar in my monorepo:

function run_tests(test_names::Union{Nothing,Vector{String}}=nothing)

    # When we pass in arguments on the command line, they're
    # read into the Base.ARGS variable. This mimics that effect
    empty!(Base.ARGS)
    if !isnothing(test_names)
        push!(Base.ARGS, "--include_tests")
        append!(Base.ARGS, test_names)
    end

    current_file_path = @__DIR__
    project_path = dirname(dirname(current_file_path))
    runtests_path = joinpath(project_path, "test", "runtests.jl")
    include(runtests_path)

    empty!(Base.ARGS)
    return nothing
end

Which lets me run things like run_tests(["db", "ml"]) and only run test sets that contain either of those strings, or run_tests() to run all tests, and works with Revise.

1 Like
3

A Julia library with the same kind of functionality that pytest offers would be fantastic. I know there are a few attempts out there but I don’t know any that are still actively maintained.

4

I think the Test Item framework is likely what your are looking for:

5

Definitely seems to support a lot of useful features. Hopefully they will keep developing it. But I will still need to use the Revise.jl flow given above.