# Transition to TestItems from a semantic Test suite, but keeping the semantics

**URL:** <https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299>\
**Category:** VS Code\
**Created:** [June 28, 2025, 12:30pm UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299 "2025-06-28T12:30:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 28, 2025, 12:30pm UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/1 "2025-06-28T12:30:41Z")

</div>

Inspired by some discussions on Slack, I am considering, to test how I could improve my current workflow to `@testitem`s, since that would give a very nice interaction in VS Code. However, I feel, that my current test setup is nearly orthogonal to `@testitem` and I can not find a good way to transition – though I by now have thought about it for quite a while.

For my setup, I would start some new tests in Manifolds.jl, but since the setup is a bit younger and hence a bit better structured in LieGroups.jl, let me used that as an example.  
But both share the fact, that they are libraries, the first of manifolds, the second of LieGroups.

In that package, we have written a Test suite, that can test single functions on a certain Lie group, for example `lie_bracket` – one of the shorter tests:

> <https://github.com/JuliaManifolds/LieGroups.jl/blob/15265899caa05d3948578bcf026f242f782b3d29/test/LieGroupsTestSuite.jl/LieGroupsTestSuite.jl#L852-L877>

and there is an overall function `test_liegroup` where you would provide such a group, and the functions one would like to test, cf. [LieGroups.jl/test/groups/test\_general\_linear\_group.jl at 15265899caa05d3948578bcf026f242f782b3d29 · JuliaManifolds/LieGroups.jl · GitHub](https://github.com/JuliaManifolds/LieGroups.jl/blob/15265899caa05d3948578bcf026f242f782b3d29/test/groups/test_general_linear_group.jl#L44), where one would before in `properties` specify functions to test and points and (Lie algebra) tangent vectors; expectations are meant for values to expect.

## To my problem:

To me `@testitem` sounds like something that would be the same order or even smaller than a `@testset`? On the other hand it fully encapsulates, so using it _within_ e.g. `test_lie_bracket` would not be possible.

On the other hand the `tags` would be great to use “from the very outside” to for example just doing a “short integration test” somewhere in other packages (further upstream).

So I can not see a way to have these single functionality-tests we currently have _together_ with test items? At least to me they feel orthogonal – I can _either_ do small encapsulated standalone tests with testitems _or_ this test suite idea, but I can not see how to combine these.

Any ideas?

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [June 28, 2025, 9:22pm UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/2 "2025-06-28T21:22:20Z")

</div>

From how your tests are structured, you could probably make each test file a test item, e.g. one test item per group, without having to change much else. You could make it more granular, but there is probably very little benefit.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 29, 2025, 9:07am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/3 "2025-06-29T09:07:05Z")

</div>

Interesting, I will give that a try; The reason I did not think about that, was that most of the docs – as well as the name “item” indicated to me that it should be more fine granular.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [June 29, 2025, 10:01am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/4 "2025-06-29T10:01:34Z")

</div>

Maybe there is a way to move most of these functions/testcode into a `@testmodule`, and then have `@testitem`s that just call these functions but don’t contain much code, actually? The `@testitem`s would be more a way to select which test code to run, but the code would mostly be in `@testmodule`s.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 29, 2025, 10:44am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/5 "2025-06-29T10:44:16Z")

</div>

The `LieGroupsTestSuite` is already a module, so maybe that could be turned into a test module?

But sure, one side could be to have a testitem per test function like the one above and with that an easy way to choose, which tests to run.

Just that for this approach I am stuck on the following:  
A `testitem` as its own environment and a very fixed setup. So I have no idea (or model in my head) yet how to use a single test item and being able to run that on different Lie groups?

Or to be more precise: To me a `testitem` reads like a monolithic set-in-stone thing that runs _exactly one test_ with _exactly one setup_.

But my `test_Lie_bracket` for example should be able to run on arbitrary Lie groups, not only one.  
That is where I am stuck with the orthogonal idea

- test item runs exactly one test on one setup
- my current functions are semantic checks of properties on a lot of setups.

I am not yet able to “mix” these two.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [June 29, 2025, 11:04am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/6 "2025-06-29T11:04:35Z")

</div>

Maybe you could put the `test_Lie_bracket` function in a `@testmodule`? And then have different `@testitem`s that exercise it with different Lie groups? Maybe one `@testitem` runs it with all groups, or maybe one `@testitem` for each group you might want to test? Not sure any of this makes sense 🙂

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 29, 2025, 11:17am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/7 "2025-06-29T11:17:03Z")

</div>

I would prefer not to have a “run this function with all groups” – because that would probably mean that someone who introduces a new group to the library would have to adapt _all_ such testitems.

And the idea of the current test suite (the current `LieGroupsTestSuite` module) is:  
If you have defined a Lie group – either a new one in the package or one somewhere in the ecosystem – specify which functions you implemented and we check they work well.  
(which is the opposite of: please add it everywhere yourself ;))

If I write one `testitem` for every group, I loose the idea of modularity, that I can use tags to choose which tests to run – which would be my second reason to switch; the first being coverage in VS code.

As I said, I am also totally clueless and lost on this, because it seems testitems provide tests in a way that is orthogonal to what I want to achieve (besides coverage in VS code).

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [June 29, 2025, 11:38am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/8 "2025-06-29T11:38:51Z")

</div>

Hm, I think really what you are looking for is a way to run a test item with a parameter value, right? If there was a way to run a test item with a custom parameter value, you could pass in the name of the group?

The VS Code API has something that might be a fit, not sure. I’ll think more about it, initially I thought we would use this for something entirely different, but maybe it could be used for this as well. But anything there will not happen anytime soon, still busy migrating the entire ecosystem to the new runner code…

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 29, 2025, 11:47am UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/9 "2025-06-29T11:47:56Z")

</div>

Well not “a” parameter value, but more like a whole set around that.

The name of the group is not enough, I need a concrete instance of that type, e.g. `SpecialOrthogonal(3)` (since it exists in arbitrary dimensions) and 1-3 points and 1-3 Lie algebra vectors and maybe other things.  
So before such a “test suite call” there is a bit of a setup as well.

And also all this here is not urgent, I already thought about it about a year ago or so, and did not find a good way to mach “my way of testing” with test items.  
And maybe the test suite idea is too different from what testitems can offer,  
then still a test item per group might be nice to get coverage in VS code at least.

The single Lie group test files are meant to be standalone anyways (i.e. can be run with `include("test/groups/test_mygroup.jl")`

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [June 29, 2025, 5:58pm UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/10 "2025-06-29T17:58:36Z")

</div>

Maybe it would be helpful to clarify why you want to switch to testitems.

For me personally, it is almost entirely about being able to run tests individually. For example, when I‘m refactoring the implementation of one unit, I want to quickly run the test for that and nothing else.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [June 29, 2025, 6:43pm UTC](https://discourse.julialang.org/t/transition-to-testitems-from-a-semantic-test-suite-but-keeping-the-semantics/130299/11 "2025-06-29T18:43:48Z")

</div>

Sure, that is a good idea.

So “a test individually” is in my setting maybe a bit ambiguous: To run the test suite on a single Lie group, making sure the functions still “interact” correctly, sure I do that often.

Running a single of the test functions like `test_lie_bracket` for a very specific Lie groups and specific point – not so often. Maybe I would do that more, if I can locally test for code cov with that.  
But I fear that would mean that the whole setup be run before, that is besides `using LIeGroups` at least

> <https://github.com/JuliaManifolds/LieGroups.jl/blob/15265899caa05d3948578bcf026f242f782b3d29/test/groups/test_general_linear_group.jl#L11-L13>

and having expectations, available like

> <https://github.com/JuliaManifolds/LieGroups.jl/blob/15265899caa05d3948578bcf026f242f782b3d29/test/groups/test_general_linear_group.jl#L41-L43>

which knows the result of the Lie bracket to expect.

In order for that to work I would probably turn all the small functions into test items(?) but I am not sure how to “switch” loading different setups then.  
Maybe the setups is really my main problem. I have a whole bunch of small tests that I want to run with (currently in this example about 18) different test scenarios. And `@testitems` are fixed to one test-module, one setup= scenario,  
so that keeps me from seeing how they would be useful for me on the fine granular level

If something like that would work, I could probably steer which tests to run with tags. That sounds nice.

Sure I could “wrap” the while test file linked about in a test item as discussed before. That brings me code cov – but nothing else of what I just said.

Well, maybe the scanarios then really do not fit. Thanks still for the inputs and clarifications.