# What's the strategy for marking a broken test that you expect to throw when it's not broken?

**URL:** https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385
**Category:** General Usage
**Tags:** testing
**Created:** [July 9, 2021, 8:08pm UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385 "2021-07-09T20:08:57Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![NHDaly](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhdaly/32/1413_2.png) [@NHDaly](https://discourse.julialang.org/u/NHDaly)
#### Post date: [July 9, 2021, 8:08pm UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385/1 "2021-07-09T20:08:57Z")

</div>

I have a function that _should_ be throwing an exception, but currently it’s not. I want to indicate that the not-throwing behavior is broken.

I tried this, but it doesn’t work, because the `@test_throws` records its failure before the `@test_broken` gets a chance to swoop in and say “no it’s fine, that was expected”:

```julia
julia> @testset "t" begin
           @test_broken @test_throws Exception 2+2 # pretend that this is supposed to throw
       end
t: Test Failed at REPL[10]:2
  Expression: 2 + 2
    Expected: Exception
  No exception thrown
Stacktrace:
 [1] macro expansion
   @ REPL[10]:2 [inlined]
 [2] macro expansion
   @ ~/builds/julia-1.6/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
 [3] top-level scope
   @ REPL[10]:2
Test Summary: | Fail Broken Total
t | 1 1 2
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 1 broken.

```

Anyone know the supported way to do this? 🙏 thanks!

---

<div class="post-metadata">

### Author: ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)
#### Post date: [July 9, 2021, 8:40pm UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385/2 "2021-07-09T20:40:25Z")

</div>

That feels like a bug to me, your code is the natural way to write what you want IMO.

---

<div class="post-metadata">

### Author: ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [July 9, 2021, 9:10pm UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385/3 "2021-07-09T21:10:28Z")

</div>

> [@jzr](#):
>
> That feels like a bug to me

Why a bug? Macros are evaluated from the innermost to the outermost and `@test_throws` doesn’t return a boolean anyways, which is what `@test_broken` would need to evaluate

---

<div class="post-metadata">

### Author: ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)
#### Post date: [July 9, 2021, 9:39pm UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385/4 "2021-07-09T21:39:37Z")

</div>

I guess I can’t speak to the original design intent, but I don’t like the noncomposability of it.

Maybe something like

```julia
julia> function throwsp(f, exceptiontype)::Bool
       try
           f()
       catch e
       if e isa exceptiontype
           return true
       end
       end
       return false
       end
throwsp (generic function with 1 method)

julia> throwsp(Exception) do 
       div(0x1,0x0)
       end
true

julia> @test_broken throwsp(Exception) do 
       1/0
       end
Test Broken
  Expression: throwsp(Exception) do 
    1 / 0
end

```

---

<div class="post-metadata">

### Author: ![NHDaly](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhdaly/32/1413_2.png) [@NHDaly](https://discourse.julialang.org/u/NHDaly)
#### Post date: [July 12, 2021, 1:47am UTC](https://discourse.julialang.org/t/whats-the-strategy-for-marking-a-broken-test-that-you-expect-to-throw-when-its-not-broken/64385/6 "2021-07-12T01:47:13Z")

</div>

FWIW I ended up writing this, which did what I want, but it’s not very satisfactory:

```julia
@test_broken (try 2+2 catch e; e; end) isa Exception

```

I like your suggestion, @jzr.

@giordana: Yeah, I know that it shouldn’t work, but it felt like maybe it should when I wrote it. Even as I wrote it, i thought “no, this isn’t going to work,” but i don’t know what i’m supposed to write instead.

I imagined/hoped that maybe the `@test_broken` would modify the context that the `@test` was operating in, to “magically” invert its result or something, like by setting a task\_local\_storage value telling `@test` not to report its result or something.

Anyway, open to other suggestions still 🙏
