Can anyone suggest how to debug break in unit test. My unit tests use standard Test.jl with @testset, @test macro and I start them from VS Code as ‘] test MyPackage’.
Thank you very much,
3 Likes
It is hard to suggest a specific action without more details. @test just gets a boolean that should be true, so usually one tries to figure out why it isn’t. You should be able to investigate the broken test in isolation.
Nothing special. I have set of tests in test folder and use Test.jl, all standard. It is usually not needed to debug them, because they are small and, if fail, it is enough to review the code. But some tests are bigger, sort of end to end, I want to debug break at some function to see values of local variables, and step through.
I usually run tests from Julia REPL in VS Code: activate my project and then do ‘] test MyProject’. There is no option to start with debugger there. I wonder if it is possible to insert some macro at some function to @debug_break, or something.
Or maybe you can advice another method, such as organize test so that they can be called from REPL as regular function (so it is easy to run them with debugger)?
Tests are code like any other Julia code, so you can just run them.
So, for example, if I have the following test, how do I run it under debugger then?
@testset "mytest" begin
# some code goes here
@test a == b
end
(note that some code may depend on something included earlier, with regular code I can just ‘using MyPackage’ and then call any function under debugger with some parameters.)
There is not much you can debug about a == b, you should debug the code that generates a and b. Eg maybe run # some code, in the debugger if you prefer.
I have the same issue. Running “Test” from the pkg REPL works. I try to run the test file in debug mode in vscode and the way it imports the main package will not work.
1 Like
Perhaps you could try TestItemRunner.jl (Prerelease of new testing framework and test run UI in VS Code). This will let you isolate your unit tests and integrate with VS code’s built in Test viewer. From there you can run individual tests. There is also usually an option to debug an individual test, but the framework is still in beta so I’m not 100% sure that feature is there. However, the runner lets you launch a specific unit test from the REPL and you can just use the standard @enter or @run (the former seems to be more reliable for me) to enter debugging mode.
the way it imports the main package will not work
is a bit unspecific, but maybe TestEnv.jl is what you’re looking for? It helps to activate the test environment and make the “extra” dependencies available (as they would be during pkg> test).
During debugging, I like to run my testes “interactive”, that is not using the pkg test environment but just open the test file as a script executing it line by line “notebook style” and observe the repl output. If something is not working, I tend to just add some @show and @info statements to the functions I am debugging and reevluate the lines in the test file until I find the bug. For me, this kind of interactive testing/programming is one of the nicest things about julia. Once you’re using the test files as a “scratch space” where you write and executed mini-tests to ensure functionality while implementing new stuff, it becomes really easy to transform them into unit tests later.
I wasn’t aware of this package. I’ll try it out, it looks promising.
I am confused about how Julia knows to import the main package in “using PackageName”. It seems it works when running the import works even with debugger, I was too lazy checking, sorry.
Thanks for suggesting that workflow, I think I’ll do that from now on.