I released a couple of new stable features for the test item framework in the last weeks. This post will describe them.
But before I dive into the new features, a quick recap what the test item framework is! The main benefit of the test item framework is that you can very easily split your tests into self-contained small test items, and then run them individually on demand. You can read my previous post about the framework here, watch a short (outdated!) YouTube demo here or take a look at the documentation here.
I should also note that the framework is no longer in preview mode. It has been used heavily for a number of packages and has been production ready for many years now. You can and should use it for your “real” projects!
Ok, and with that, here are the new features:
Sharing code across @testitems
By default @testitems do not share any code between each other and have no dependencies between each other. These properties make it feasible to run @testitems by themselves, but sometimes one wants to share common code between multiple @testitems. The test item framework provides two macros for this purpse: @testsnippet and @testmodule. These two macros can appear in any .jl file in a package.
Test snippets
A @testsnippet is a block of code that individual @testitems can run before their own code runs. If a @testitem takes a dependency on a particular @testsnippet, that snippet will run every time the @testitem runs.
The definition of a @testsnippet might look like this
@testsnippet MySnippet begin
foo = "Hello world"
end
A @testitem can utilize this snippet by using the setup keyword like this:
@testitem "My test item" setup=[MySnippet] begin
@test foo == "Hello world"
end
Test modules
A @testmodule defines a Julia module that can be accessed from @testitems. Such a module will only be run once per Julia test process. If for example two @testitems depend on a @testmodule, it will only be evaluated once, and then the entire module will be made available to both @testitems.
The definition of a @testmodule might look like this
@testmodule MyModule begin
foo = "Hello world"
end
A @testitem can utilize this module by again using the setup keyword. Unlike with @testsnippets, the content of a @testmodule is run inside a regular Julia module, so to access content inside there one needs to prefix the module name to any name defined in the test module. A @testitem that utilizes the @testmodule just defined might look like this:
@testitem "My test item" setup=[MyModule] begin
@test MyModule.foo == "Hello world"
end
Note how we access foo with the expression MyModule.foo here.
Debugging of @testitems
@testitems can be run in the debugger by launching them via the Debug Test command. This command can be access in various places in the VS Code UI. In the test main testing view it is available here:
One can also right click on the run test icon in the text editor to select the debug option:
When a test item is run in the debugger, one can set breakpoints both in the code that is being tested or in the @testitem itself and then utilize all the regular features of the Julia VS Code debugger.
Code coverage
On Julia 1.11 and newer one can run test items in a code coverage mode and display code coverage results directly in VS Code.
To run test items in code coverage mode one launches them with the command Run Tests with Coverage. This command is availble both in the main testing view
as well as in the context menu in the text editor:
The coverage results are then displayed in various ways in the VS Code UI. For example a summary view shows coverage per file:
One can see detailed line coverage information inside the text editor:
Coverage results are also displayed inline in the regular explorer part of the VS Code UI.
Documentation
I wrote documentation 
For the whole thing, mostly copy paste from this and past discourse announcements, you can find it here.
