Getting performance information from testsets

General Usage
1

Testsets know already how much time they took to execute:

using Test

ts = @testset "outer" begin
    sleep(0.1)
end

# ts.time_end - ts.time_start = 0.10119009017944336

However, I would like to also know some other information like allocations and compile times etc.
I have a few hundred testsets and would like to programmatically do things like search for the testset that had the highest total_time - compile_time etc.

What is the best way to do this? Can it be done without annotating every single testset?

2

Probably the easiest idea is to wrote your own macro that expands to @testset and just wraps it in some code that measures whatever performance indicators you find interesting plus some code to add that to some global bookkeeping datastructure such that you can query the information later. Then you do not need to annotate all testsets yourself but just use a different macro instead of @testset.

1 Like