# Nesting @testset; good idea, or not?

**URL:** <https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164>\
**Category:** General Usage\
**Tags:** testing\
**Created:** [February 18, 2017, 2:09am UTC](https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164 "2017-02-18T02:09:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [February 18, 2017, 2:09am UTC](https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164/1 "2017-02-18T02:09:12Z")

</div>

I see two common patterns when it comes to testing larger packages.

## Coarse Style

### runtests.jl (Coarse Style)

(this is often done with a loop over filenames but I’ll write it here in full)

```julia
using Base.Test
@testset "compA" begin
    include(compA.jl)
end

@testset "compB" begin
    include(compB.jl)
end

```

### compA.jl and similarly compB.jl (Course Style)

```julia
using Base.Test #reuse this in the included file so can run standalone

# Tests for foo
@test foo(1)==7
@test foo(2)==9
@test foo(3)==16

# Tests for bar
@test bar(1)==8
@test bar(2)==-8

```

##Fine Style

### runtests.jl (FineStyle)

(this is also often done with a loop over filenames but I’ll write it here in full)

```julia
using Base.Test
include(compA.jl)
include(compB.jl)

```

### compA.jl and similarly compB.jl (Fine Style)

```julia
using Base.Test #reuse this in the included file so can run standalone
@testset "foo" begin
    @test foo(1)==7 
    @test foo(2)==9
    @test foo(3)==16
end

@testset "bar" begin
    @test bar(1)==8
    @test bar(2)==-8
end

```

* * *

Are there any pros or cons to combining them?  
That is to say taking `runtest.jl` from the coarse style, and `compA.jl` and `compB.jl` from the fine style.  
I know it is legal.  
As I recall what happens when you do `julia runtest.jl`, is that you only see the records for the outer most layer – that is the coarse `runtests.jl`.  
But it fully includes the total counts from the lower levels.

Seems like it is purely, since now if running a subset of the tests eg `julia compA.jl`, we would also get a nice breakdown table; at the fine grained level.

But I’ve never seen this done in packages.  
So perhaps I am wrong and it doesn’t work so good

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 18, 2017, 3:37am UTC](https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164/2 "2017-02-18T03:37:23Z")

</div>

> [@oxinabox](#):
>
> But I’ve never seen this done in packages.So perhaps I am wrong and it doesn’t work so good

I use the course style because I just want to know which test failed, and the line number.

> <https://github.com/SciML/OrdinaryDiffEq.jl/blob/master/test/runtests.jl>

This setup gives me all of the information that I need, is easy to extend, and separates tests of different features well.

What I like about it too is that I can keep all of my test files as examples, and just develop through tests. For example

[https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/test/ode/ode\_event\_tests.jl](https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/test/ode/ode_event_tests.jl)

This looks like any other script, but with a few `@test`s around. But if you run it, it’ll just work. And you can see I commented out common plot commands I use. I develop it interactively in that file, and then add the test when it’s complete.

If I wanted to do the “Fine style”, then I would have to gather the tests, or put a big testset around it. Juno will by default be funny with the big block, and inline execution will try to run the whole block every time, so that’s a no-go. Secondly, I’d have to group a lot of my terms if I wanted to use the same names. So just making the whole file “normal” is just easier for interactivity I think.

---

<div class="post-metadata">

**Author:** ![ssfrr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ssfrr/32/3736_2.png) [@ssfrr](https://discourse.julialang.org/u/ssfrr)\
**Post date:** [February 20, 2017, 6:13pm UTC](https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164/3 "2017-02-20T18:13:13Z")

</div>

I definitely nest my testsets, Generally a top-level `@testset "MyPkg Tests" begin ... end` that just includes all the test files, then each test file includes a top-level `@testset`, then each test case within that has its own `@testset`. Example [here](https://github.com/JuliaAudio/SampledSignals.jl/blob/38289706e65478f3b0af10c9a73cbba6544f105e/test/runtests.jl).

I generally think of my inner-most `@testset`s as test cases that generally only have a couple `@test` assertions within them. That way I can name what each test is doing pretty granularly. If everything passes it only displays the top-level statistics, but if there are any failures within then it should show the whole testset tree down to the one that failed. The indentation was messed up for a while so it was a little harder to see the nesting, but that’s been fixed recently.

---

<div class="post-metadata">

**Author:** ![fengyang.wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fengyang.wang/32/104_2.png) [@fengyang.wang](https://discourse.julialang.org/u/fengyang.wang)\
**Post date:** [March 16, 2017, 6:35am UTC](https://discourse.julialang.org/t/nesting-testset-good-idea-or-not/2164/4 "2017-03-16T06:35:49Z")

</div>

I’ve lately tried the

```julia
@testset "foo" begin
    include("test.jl")
end

```

style, and found unfortunately that line number information seems to be destroyed by this style. Has anyone else found similar behaviour? What workarounds exist?