Test cases and isolation

General Usage
1

Hi All,

Recently I was writing some test cases for the PDF library I am developing but wanted to isolate test cases in a manner that failure of one test case should not affect the subsequent tests significantly. My code is given below for reference.

  @testset "Test FlateDecode" begin
    @test begin
      filename="files/1.pdf"
      println(filename)
      doc = pdDocOpen(filename)
      @assert pdDocGetPageCount(doc) == 2
      page = pdDocGetPage(doc, 1)
      @assert pdPageIsEmpty(page) == false
      contents = pdPageGetContents(page)
      bufstm = get(contents)
      buf = read(bufstm)
      close(bufstm)
      @assert length(buf) == 18669
      @assert length(pdPageGetContentObjects(page).objs)==190
      pdDocClose(doc)
      length(utilPrintOpenFiles())==0
    end
  end

When I try to test a particular task for a specific file I will need to cover other aspects as well and they cannot be completely excluded. For example, file has to be opened. Text has to be taken out and then processing has to be carried out. At the same time any exception raised in the test blocks have to be kept within a @test block so that subsequent tests are not affected. But exceptions can be raised in non-test blocks as well leading to confusion on what didn’t work. Actually, test conditions are wrapped in @assert blocks in this case. The only concern is there will not be any failures but just errors. Lastly from a reporting standpoint multiple @test creates artificial bloating of number of tests while typically one block is kind of one test case.

I am interested to know how others are able to tackle this.

Moreover, @test macro does not have a label. This is limiting as I feel a label can provide some documentation to the test scenario in a verbose mode of reporting later.

regards,

Sambit

1 Like