# My unit tests are interferring with each other

**URL:** <https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686>\
**Category:** New to Julia\
**Created:** [March 9, 2019, 10:04pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686 "2019-03-09T22:04:57Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [March 9, 2019, 10:04pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/1 "2019-03-09T22:04:57Z")

</div>

I got a large module and to make sure everything works, I have created more than forty unit tests. But now they started to interfere with each other.

I have produce a smallest working example below to explain what my problem is.

File: runtests.jl

```julia
using Test

function tests()
    @testset "All tests" begin
        include("test_bar.jl")
        include("test_foo.jl")
    end
end

tests()

```

File: test\_bar.jl

```julia
using Test

func1(x::Int64) = x^2

function bar(x,f)
    f(x)
end

@testset "bar function" begin

code = :( bar(2,func1) ); result = eval(code)
println("$code = ", result, " ", @test result == 4 )

end

```

File: test\_foo.jl

```julia
using Test

func1(x) = x^3

function foo(x,f)
    f(x)
end

@testset "foo function" begin

code = :( foo(2,func1) ); result = eval(code)
println("$code = ", result, " ", @test result == 8 )

end

```

Individually each test works perfectly but when I execute “runtests.jl” this is the output

Output of runtests.jl

```julia
bar(2,func1) = 4 Test Passed
foo function: Test Failed at /Users/ssiew/juliascript/test_prob_dir/test_foo.jl:12
  Expression: result == 8
   Evaluated: 4 == 8
Stacktrace:
 [1] macro expansion at /Users/ssiew/juliascript/test_prob_dir/test_foo.jl:12 [inlined]
 [2] macro expansion at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [3] top-level scope at /Users/ssiew/juliascript/test_prob_dir/test_foo.jl:11
foo(2,func1) = 4 (Test Failed at /Users/ssiew/juliascript/test_prob_dir/test_foo.jl:12
  Expression: result == 8
   Evaluated: 4 == 8, Union{Ptr{Nothing}, InterpreterIP}[Ptr{Nothing} @0x000000011d86cac8, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000011d869ad4, Ptr{Nothing} @0x000000011d86bf95, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000010b179be2, Ptr{Nothing} @0x000000010b154d7f, Ptr{Nothing} @0x000000010b17a7d3, Ptr{Nothing} @0x000000010e275670, Ptr{Nothing} @0x000000010e2800bc, Ptr{Nothing} @0x000000011d85fd67, Ptr{Nothing} @0x000000011d85ed44, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000010b30daa8, Ptr{Nothing} @0x000000010b30c59c, Ptr{Nothing} @0x000000010b30cae8, InterpreterIP(CodeInfo(
 1 ─ %1 = tests() │
 └── return %1 │
), 0x0000000000000000), Ptr{Nothing} @0x000000010b30cb40, Ptr{Nothing} @0x000000010b1796cd, Ptr{Nothing} @0x000000010b154d7f, Ptr{Nothing} @0x000000010e2aeabd, Ptr{Nothing} @0x000000011d85694f, Ptr{Nothing} @0x000000011d85d252, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000011d85720f, Ptr{Nothing} @0x000000011d855c5c, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000011d844aa7, Ptr{Nothing} @0x000000010b148983, Ptr{Nothing} @0x000000011d831d1f, Ptr{Nothing} @0x000000011d83244d, Ptr{Nothing} @0x000000010b159372, Ptr{Nothing} @0x000000011d82eff0, Ptr{Nothing} @0x000000011d82f075, Ptr{Nothing} @0x000000010b163b70])
Test Summary: | Pass Fail Total
All tests | 1 1 2
  bar function | 1 1
  foo function | 1 1
ERROR: LoadError: Some tests did not pass: 1 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /Users/ssiew/juliascript/test_prob_dir/runtests.jl:10
julia> 

```

What should I do to fix this?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [March 9, 2019, 10:23pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/2 "2019-03-09T22:23:35Z")

</div>

You should reorganize your unit tests so that you don’t overwrite definitions. After some iterations writing unit tests, I came up with a very standard approach in all my packages. You can check this runtests.jl file for an example: [GeoStatsDevTools.jl/runtests.jl at master · juliohm/GeoStatsDevTools.jl · GitHub](https://github.com/juliohm/GeoStatsDevTools.jl/blob/master/test/runtests.jl)

Basically you load all the packages once, and define common functions at a top level at runtests.jl and then include small test files for specific concepts, all wrapped in `@testset`.

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [March 9, 2019, 10:31pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/3 "2019-03-09T22:31:19Z")

</div>

Is there not a way that I could make “func1” local only to the unit test and not appear in another unit test? I tried the keyword local but it does not work.

I would prefer not to define func1 on the runtests.jl because I would like to see the definition of func1 in the unittest that it is used in.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 9, 2019, 10:36pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/4 "2019-03-09T22:36:52Z")

</div>

`include` is the issue here. This library fixes it with `@safetestset`

> **[GitHub - YingboMa/SafeTestsets.jl](https://github.com/YingboMa/SafeTestsets.jl)**
>
> Contribute to YingboMa/SafeTestsets.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [March 9, 2019, 10:42pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/5 "2019-03-09T22:42:51Z")

</div>

Chris, How does running it inside a module using SafeTestsets.jl fixes the problem? func1 defined in unittest A would still be seen by unittest B

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 9, 2019, 10:45pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/6 "2019-03-09T22:45:15Z")

</div>

No, the `include`d file is in a module, so it’s not added to `Main`, so the two `func1`s are different functions.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [March 9, 2019, 10:49pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/7 "2019-03-09T22:49:39Z")

</div>

SafeTests seems interesting. I was not aware of it. As an alternative, I modeled my tests after the tests in [Distributions.jl](https://github.com/JuliaStats/Distributions.jl/blob/master/test/runtests.jl), which loops through each file and calls `@eval module $(Symbol("Test_", t))` to ensure that each test is conducted in an isolated scope.

---

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [March 9, 2019, 10:56pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/8 "2019-03-09T22:56:31Z")

</div>

Christ Fisher

Yes that does looks interesting. But why is there a return keyword in it?

File: runtests.jl

```julia
using Test

alltests = ["bar","foo"]

function tests()
    @testset "All tests" begin   
        res = map(alltests) do testname
            @eval module $(Symbol("testmodule_", testname))
                include("test_" * $testname * ".jl")
            end
            return # WHY IS THERE A RETURN KEYWORD HERE
        end
    end
end

tests()

```

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [March 9, 2019, 11:11pm UTC](https://discourse.julialang.org/t/my-unit-tests-are-interferring-with-each-other/21686/9 "2019-03-09T23:11:45Z")

</div>

I think it is equivalent to `return nothing`, which prevents the function from returning the most recent result.