# Why is \`Test\` so meta-programming heavy? **URL:** \ **Category:** Internals & Design\ **Tags:** testing, design\ **Created:** [February 14, 2023, 11:00am UTC](https://discourse.julialang.org/t/why-is-test-so-meta-programming-heavy/94619 "2023-02-14T11:00:03Z")\ **Posts on this page:** 1\ **Showing post:** 4

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\ **Post date:** [February 14, 2023, 2:38pm UTC](https://discourse.julialang.org/t/why-is-test-so-meta-programming-heavy/94619/4 "2023-02-14T14:38:38Z")

> [@nrontsis](#): > > Why do you say that metaprogramming is required for this aggregation? Given a vector of `test_*` functions one can do the same by redirecting stdout/stderr, no? For example `pytest` achieves nice aggregation without metaprogramming as far as I am aware. I think one of the core ideas behind testing in Julia is that `test/runtests.jl` should be a simple julia script that runs all tests when it is executed (either via `include()` in an existing session, or running `julia test/runtests.jl` in the command line, or via `Pkg.test()` which sets up a “hardened” context to catch more errors). My understanding is that this idea is reflected in the design of the `Base.Tests` standard library, where it is assumed that tests are plain scripts, and evaluating `@testset` stanzas should actually run the tests inside. This contrasts with the approach taken in `pytest` (as I understand it), where tests are defined as functions following a certain naming convention. This means that evaluating them only defines the function, but does not execute any code; in order to actually run the tests, you need another, specific entry point, that does a bit of introspection to find all functions defining tests and actually call them. This preliminary stage presumably allows `pytest` to nicely organize its outpout. However, since Julia’s standard `Base.Test` approach functions as a plain script execution, it can’t really perform such a kind of a global preliminary analysis… except if at some stage it has access to a whole set of tests defintions thanks to the `@testset` macro. Maybe the macros could have been replaced by higher-order functions, but I doubt this would have produced a system where stack traces are more legible. > [@nrontsis](#): > > In my mind there could be much simpler alternatives like e.g. following something similar to [`pytest`](https://github.com/pytest-dev/pytest) (but without its complicated `fixtures`) so that unit tests are simply all the functions named `test_*` in the `test` folder of a package, organised in groups by their path. Are you aware of [`TestItemRunner`](https://github.com/julia-vscode/TestItemRunner.jl) and the new unit tests UI in VS Code? It might be a better fit for your workflow (don’t mind the `@testitem` high-level macro: AFAIU it merely acts as an annotation identifying test definitions, but is not really macro-expanded) > [@Prerelease of new testing framework and test run UI in VS Code](https://discourse.julialang.org/t/prerelease-of-new-testing-framework-and-test-run-ui-in-vs-code/86355): > > I just pushed out a new release of the Julia VS Code extension (1.7.6) to everyone that includes a preview of our new test UI and a general new testing framework for Julia. For a short demo, take a look at our Juliacon talk [https://youtu.be/Okn\_HKihWn8?t=1268](https://youtu.be/Okn_HKihWn8?t=1268). This new set of features consists of two new packages ([TestItems.jl](https://github.com/julia-vscode/TestItems.jl) and [TestItemRunner.jl](https://github.com/julia-vscode/TestItemRunner.jl)) and the new UI in the Julia VS Code extension. A good way to refer to this new test framework is by the “test item” name. How to write test items … --- _[View the full topic](https://discourse.julialang.org/t/why-is-test-so-meta-programming-heavy/94619)._