Suppress Stacktrace when a test fails

General Usage
1

Is there a way to suppress Stacktrace output when a test fails when using @test macro? I just want to know which test failed and what the expected output was and what the actual output was. The @test macro provides this information. But it also provides a lot of additional output which contains Stacktrace and even LoadError (which I am not sure why it is throwing). The additional output is distracting for the purpose that I want to use it for. So, is there a way to suppress it, that is not too difficult?

Thank you.

1 Like
2

This is not really controllable in a simple way (not by design, but simply by happenstance — Test could be improved in several ways; the way it collects and prints test results being one such way).

An easy/quick/horrible workaround is to monkey patch record(::DefaultTestSet, ::Union{Fail, Error}):

julia> using Test

julia> @testset "asdf" begin
           @test 0 == 1
       end
asdf: Test Failed at REPL[2]:2
  Expression: 0 == 1
   Evaluated: 0 == 1
Stacktrace:
 [1] top-level scope at REPL[2]:2
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Test/src/Test.jl:1083
 [3] top-level scope at REPL[2]:2
Test Summary: | Fail  Total
asdf          |    1      1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.


julia> Test.eval(quote
       function record(ts::DefaultTestSet, t::Union{Fail, Error})
           if myid() == 1
               printstyled(ts.description, ": ", color=:white)
               # don't print for interrupted tests
               if !(t isa Error) || t.test_type != :test_interrupted
                   print(t)
                   # don't print the backtrace for Errors because it gets printed in the show
                   # method
                   if !isa(t, Error)
                       # Base.show_backtrace(stdout, scrub_backtrace(backtrace())) # Commented out
                   end
                   println()
               end
           end
           push!(ts.results, t)
           t, isa(t, Error) || backtrace()
       end
       end)

julia> @testset "asdf" begin
           @test 0 == 1
       end
asdf: Test Failed at REPL[4]:2
  Expression: 0 == 1
   Evaluated: 0 == 1
Test Summary: | Fail  Total
asdf          |    1      1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.

An arguably saner alternative is to implement your own test set type. That’s a fair bit of work though.

The best option would be a big cleanup to how stdlib/Test deals with capturing and printing test results. That’s quite a lot of work.

4 Likes
3

Thank you for the reply. Will take me some time to go through your reply. But it seems it is something I will let alone for now given the time constraints and my current level of Julia knowledge.

4

The short story: if you execute the Test.eval bit, this should do what you wanted (this very slightly modifies the code inside stdlib/Test at runtime). However, it depends on internal implementation details of Test and will probably stop working in a future Julia release.

5

Thanks so much. I will try to see if that works. It it does, that would be great.

6

Is there any news on this? This seems so useful, it would be nice to have a way of doing tests that gave clean output without stacktraces, when desired. I don’t understand what’s going on well enough to dig into it myself though.

1 Like
7

I think this simple enough, easy enough, and clean enough

Test.eval(quote
	function record(ts::DefaultTestSet, t::Union{Fail, Error})
		push!(ts.results, t)
	end
end)
2 Likes