Macro Expansion Error in basic @testset

General Usage
,
1

Anytime I get something that evaluates false inside of a testset I get a macro expansion error. The testset runs as expected and gives a summary as expected, but throws the macro expansion error first, and I am a bit unsure why. I don’t think it will affect my testing, but I just want to check. Here is the most basic example

using Test
@testset "Macro expansion error" begin
    @test false
    @test true
end

I feel like maybe I messed something up somewhere, but I deleted the REPL and started a new one (I am using VS Code) and it didn’t make a difference.
Here is the output

Macro expansion error: Test Failed at REPL[4]:2
  Expression: false
Stacktrace:
 [1] macro expansion
   @ C:\Users\child\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Test\src\Test.jl:445 [inlined]
 [2] macro expansion
   @ REPL[4]:2 [inlined]
 [3] macro expansion
   @ C:\Users\child\AppData\Local\Programs\Julia-1.7.1\share\julia\stdlib\v1.7\Test\src\Test.jl:1283 [inlined]
 [4] top-level scope
   @ REPL[4]:2
Test Summary:         | Pass  Fail  Total
Macro expansion error |    1     1      2
ERROR: Some tests did not pass: 1 passed, 1 failed, 0 errored, 0 broken.

The tests still run, so if it is not a problem I will move forward, I just wanted to know.
Thanks so much!
Jacob

2

Are you referring to the part of the output that says Macro expansion error? That’s just the value of the string that you wrote after @testset. There is no actual macro expansion error. All of the macro expansion entries in the stacktrace are a bit noisy, but what you really care about is the location in your source code of the failed test.

Here is a somewhat more realistic example. Suppose you have a file called test_foo.jl that contains the following code:

using Test

bar() = 1 
foo(x) = 2x

@testset "hello world" begin
    @test foo(1) == 3
end

Now in the REPL we observe the following if we include that file:

julia> include("test_foo.jl")
hello world: Test Failed at /Users/cameron/test_foo.jl:7
  Expression: foo(1) == 3
   Evaluated: 2 == 3
Stacktrace:
 [1] macro expansion
   @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/test_foo.jl:7 [inlined]
 [3] macro expansion
   @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/test_foo.jl:7
Test Summary: | Fail  Total  Time
hello world   |    1      1  0.4s
ERROR: LoadError: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /Users/cameron/test_foo.jl:6

The location of the failed test is given at the top where it says

Test Failed at /Users/cameron/test_foo.jl:7

You can also see the location of the failed test by looking at the fourth entry in the stacktrace:

@ ~/test_foo.jl:7

A failing @test will always throw an exception, even when it is inside a @testset. The nice thing about @testset is that it gathers all the thrown exceptions from failed tests that occur inside it, rather than failing on the first failed test. However, any exception inside a @testset other than a failed test will immediately halt the test set, as we can see in the following example:

julia> @testset "hello world" begin
           sqrt(-1)
           @test false
           @test true
           @test false
       end
hello world: Error During Test at REPL[21]:1
  Got exception outside of a @test
  DomainError with -1.0:
  sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
  Stacktrace:
    [1] throw_complex_domainerror(f::Symbol, x::Float64)
      @ Base.Math ./math.jl:33
    [2] sqrt
      @ ./math.jl:591 [inlined]
    [3] sqrt(x::Int64)
      @ Base.Math ./math.jl:1372
    # ... Removed for clarity ...
Test Summary: | Error  Total  Time
hello world   |     1      1  0.0s
ERROR: Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken.
2 Likes
3

Thanks so much! I guess I misunderstood the stack trace macro expansion, I thought it was saying I had a legitimate error, but from your explanation I think I am understanding that it is just a thrown exception for a failed test and a trace to where it was. Also it is good to know that normal errors still stop the code inside of a test set. Thanks for the in depth answer!
Thank you!

1 Like