# How to test that a testset fails expectedly?

**URL:** <https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259>\
**Category:** General Usage\
**Created:** [December 1, 2025, 2:53pm UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259 "2025-12-01T14:53:37Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 1, 2025, 2:53pm UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/1 "2025-12-01T14:53:37Z")

</div>

Let’s say I have a package that defines a test macro. The macro creates some code that runs a `@testset` and inside a couple `@test`s. If I want to test that the macro works, I have to test that it succeeds if the conditions are right and that it fails when the conditions are wrong.

To test that it succeeds, I can just use it:

```julia
@testset "test that things work" begin
    @my_test some + successful + code
end

```

Now my question is, how do I test that the inner testset correctly fails when it’s supposed to? This doesn’t work because the function doesn’t actually throw, but it the spirit of what I’m trying to do:

```julia
@testset "test that things fail" begin
    @test_throws TestFailure @my_test some + failing + code
end

```

If I do something with the TestSet returned from `@my_test` I still think I can’t keep the outer testset from failing due to the way Test works internally, I can’t tell the outer testset to ignore the inner.

---

<div class="post-metadata">

**Author:** ![asinghvi17](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asinghvi17/32/8272_2.png) [@asinghvi17](https://discourse.julialang.org/u/asinghvi17)\
**Post date:** [December 1, 2025, 8:00pm UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/2 "2025-12-01T20:00:28Z")

</div>

The global state makes things pretty dicey there.

Perhaps you could create [your own testset type](https://docs.julialang.org/en/v1/stdlib/Test/#Creating-Custom-AbstractTestSet-Types) that would do the right thing on the failure of an inner testset? Probably by customizing `Test.record` somehow. That `@customtestset` could then wrap an invocation of your testing macro.

---

<div class="post-metadata">

**Author:** ![adolgert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adolgert/32/20286_2.png) [@adolgert](https://discourse.julialang.org/u/adolgert)\
**Post date:** [December 3, 2025, 11:03am UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/3 "2025-12-03T11:03:47Z")

</div>

More ideas:

- You wrote a `@my_test` macro. You could implement the macro by having it do most of its work in a function. Then use the function in the failing test. That would test most of the functionality.
- And a nice way to test macros is to use @macroexpand. So you test expansion, not execution.
- And this is silly, but you could use `run(pipeline(`julia -e $code`, stderr=devnull), wait=false)` to run a tiny testset that fails and test the `exitcode` to see that it failed. It would work for this one test.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 3, 2025, 11:13am UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/4 "2025-12-03T11:13:09Z")

</div>

Thanks for the suggestions, what I did in the end was to write a variant of the macro reusing most of the code that explicitly tests the failure case. Seems what I wanted to do was not possible the way Test.jl is designed

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [December 3, 2025, 12:08pm UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/5 "2025-12-03T12:08:21Z")

</div>

There is [GitHub - JuliaTesting/MetaTesting.jl: Test the testers](https://github.com/JuliaTesting/MetaTesting.jl) which I think is exactly for this purpose

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 3, 2025, 12:22pm UTC](https://discourse.julialang.org/t/how-to-test-that-a-testset-fails-expectedly/134259/6 "2025-12-03T12:22:03Z")

</div>

Nice that looks exactly like what I was thinking about