a reproducible environment
Off-topic I thought that @safetestset was needed for reproducibility?
Sure, it could have been a function instead, but then you need lots and lots more global variables to synchronize stuff, or your testing framework becomes much less flexible (by actually shoving all that complexity into each and every test function)
I am afraid I am not following why these are necessary. Perhaps I am missing something?
Out of interest, have you looked at how
Testworks internally and what it tries to achieve? It’s not using metaprogramming just for the heck of it, but giving a uniform interface for allowing outside users to very easily hook into the existing@testsetmachinery with their own testsets, for recording & displaying purposes.
I am afraid that I don’t know much about the internals of Test. What I still don’t understand is why is meta-programming is necessary. I have seen quite large test suites work without it (e.g. via pytest as I mentioned).
So while I am still puzzled about what are the advantages of a metaprogramming-heavy test framework are, I know some of it’s current limitations, most notably (for me): loosing the ability to use a debugger.
An example of that would be appreciated, since that sounds more like an issue with errors in generated functions than
@testsetper se.
Sorry, I gave it a go, and could not generate a minimal example for this 
What I have observed was getting a single line stacktrace (that was of very little use) when running within a @testset as compared to a full stacktrace when without, with regards to a bug of a generated function of some library. I agree, perhaps it was not an issue of @testset itself - just not a great user experience.