[ANN] TestPicker - Interactive tool for testing based on fzf and bat

Package Announcements
, ,
1

We all know testing does not necessarily has the best support in Julia. And if there are great frameworks like ReTest.jl or TestItemRunner.jl to mention these only, they still rely on replacing the traditional Test.jl framework.

TestPicker.jl does not try to achieve anything new but to make running tests, especially in large repositories a bit easier.

A demo is worth a thousand characters so here is one!

asciicast

The main features TestPicker brings are:

  • Running all tests with the right test environment (reusing TestEnv.jl).
  • Tests are run in a module and do not affect the Main environment.
  • Ability to run single testsets, even nested ones by fetching the right preamble from the file.
  • Selection of one or multiple testfile/testset using fzf for an interactive experience associated with a highlighted preview of the content using bat
  • Everything embedded in a repl mode (activated with !). I am aware that this potentially collide with some code, I hope to find a better solution in the future.
  • Quick shortcut test> - to repeat the last set of tests
  • Preview of failed tests using fzf again with a quick preview on the stack traces/failed test information.
  • Preview of stack traces with source preview of the selected trace.

One main constraint of the tool is that test files should be self-contained (which I believe is a better practice), i.e. each file should contain all the using ... statements or include("test-tooling.jl") it needs.

Give it a try and please report any issue you face! :hugs:

28 Likes
2

Yup, this is going into my global environment. This looks super. Thanks for sharing and the creation @theogf !

3 Likes
3

Is this compatible with using TestItems?

4

Not per se. If a @testitem macro is used and using TestItems is present in the file, running a test file would work.
However @testitem blocks will not be identified directly (as for @testset). It should actually be possible to do it so I might give it a try.

5

Maybe more generally there can be a configuration option to manually add to the preamble or have a default tests/testpicker_preamble.jl that is always loaded if found?

6

I actually tried that already because I wanted to try out TestPicker.jl in a project where I use TestItems.jl. Turned out that also checking for Symbol(@testitem) here was enough to make it work. Would be cool if that would be directly in the package. It would even be better if one could add other symbols (or search for patterns like testset as substring) that are counted as testset because, e.g., in Trixi.jl we use a custom @trixi_testset.

3 Likes
7

I would be happy to let users define their own symbols to be detected but I wonder how to properly capture those.
For now I will do the dumb thing and just ENV but we can get something fancier and more robust eventually.

e.g., in Trixi.jl we use a custom @trixi_testset.

Nice package btw!

1 Like
8

Thanks! The ENV solution is already quite nice.