Inspired by some discussions on Slack, I am considering, to test how I could improve my current workflow to @testitems, since that would give a very nice interaction in VS Code. However, I feel, that my current test setup is nearly orthogonal to @testitem and I can not find a good way to transition – though I by now have thought about it for quite a while.
For my setup, I would start some new tests in Manifolds.jl, but since the setup is a bit younger and hence a bit better structured in LieGroups.jl, let me used that as an example.
But both share the fact, that they are libraries, the first of manifolds, the second of LieGroups.
In that package, we have written a Test suite, that can test single functions on a certain Lie group, for example lie_bracket – one of the shorter tests:
and there is an overall function test_liegroup where you would provide such a group, and the functions one would like to test, cf. LieGroups.jl/test/groups/test_general_linear_group.jl at 15265899caa05d3948578bcf026f242f782b3d29 · JuliaManifolds/LieGroups.jl · GitHub, where one would before in properties specify functions to test and points and (Lie algebra) tangent vectors; expectations are meant for values to expect.
To my problem:
To me @testitem sounds like something that would be the same order or even smaller than a @testset? On the other hand it fully encapsulates, so using it within e.g. test_lie_bracket would not be possible.
On the other hand the tags would be great to use “from the very outside” to for example just doing a “short integration test” somewhere in other packages (further upstream).
So I can not see a way to have these single functionality-tests we currently have together with test items? At least to me they feel orthogonal – I can either do small encapsulated standalone tests with testitems or this test suite idea, but I can not see how to combine these.
Any ideas?