Using TestItemRunner snippets & modules when troubleshooting

I’ve been starting to use TestItemRunner’s new @testsnippet and @testmodule mechanisms to help organize testing code, and I’ve found it very helpful.

One thing that’s a setback, though, is that if a test fails and I need to do some troubleshooting, it’s not clear how to proceed.

  • Setting a VS Code breakpoint generally doesn’t work - it did work for a little while after the feature was released a couple months ago, but these days all my tests either crash when I hit the debugger button[1], or they just don’t stop at the breakpoint.
  • I tried @run_package_tests filter = ti -> contains(ti.filename, "my_test") from the REPL, but this generally crashes too[2].
  • Copying & pasting the test code into my REPL, which was my previous go-to workaround, doesn’t work anymore because it now requires the setup module/snippet to function.

Does anyone have any recommendations for this?


[1] error generally looks like The test process crashed while running this test, and stack trace shows @ Main ~/.vscode/extensions/julialang.language-julia-1.127.2/scripts/error_handler.jl:15 ERROR: Endpoint is not running, the current state is closed.

[2] error generally looks like LoadError: UndefVarError: 'Foo' not defined, where ‘Foo’ is the name of the package being tested. Even if I put using Foo in the test.

2 Likes

Just a ping - I’m still interested in hearing how/whether other people deal with this part of their development workflow. Thanks.

Related to this, a few days ago I opened an issue in TestItems.jl asking for the possibility to include a @testmodule as a standard module.

My use case is exactly the same as in the OP, and I didn’t find a good solution for it.

1 Like