@test return is not what the docs say it is

Internals & Design
, ,
1

The documentation for @test suggests that if the expression returns either true of false, you should get back either Test.Pass or Test.Fail. However, running simply foo = @test false in the repl causes and error (so foo isn’t actually defined), and running it in a test set:

@testset "foo" begin
           thing = @test false
           println(typeof(thing))
end

Gives a Tuple{Test.Fail,Array{Union{Ptr{Nothing}, Base.InterpreterIP},1}}. So it seems there’s a discrepancy between the documentation and the actual behavior.

1 Like
3

I guess the use of the word “returns” is in error, it probably should say “prints”. Is there something specific you want to do? If you need the result of a previous test so the following tests only execute under a successful situation you can do:

@testset "foo" being
    thing = false
    @test thing = ....
    if thing == true
         # additional tests
    end
end
1 Like
4

I’m putting several tests inside of a for loop, and I would like to know for which values of the iteration the tests fail. Your suggested workaround will definitely work for me

@testset "foo" being
    for ii=1:10
      thing = false
      someResult = functionOfII(ii)
      @test thing = ...# compare someResult to something else
      if !thing
            # print the value of ii
      end
    end
end
5

If you’re not doing anything other than printing the result, you can get a similar effect by nesting testsets. E.g.

@testset "foo" for i in 1:10
    @testset "Test $i" begin
        @test iseven(i)
    end 
end
5 Likes
6

Nesting is not actually necessary. You can do

@testset "foo $i" for i in 1:10
        @test iseven(i)
end

I think nesting is more useful the other way around (for loop in the inside testset):

@testset "foo" begin
    @testset "Test $i" for i in 2:2:10
        @test iseven(i)
    end 
end

This way, if all tests succeed the summary is in one line, and if a test fails you still know which one.

3 Likes