# How to improve include() performance in runtests.jl?

**URL:** <https://discourse.julialang.org/t/how-to-improve-include-performance-in-runtests-jl/56987>\
**Category:** Performance\
**Created:** [March 12, 2021, 12:48am UTC](https://discourse.julialang.org/t/how-to-improve-include-performance-in-runtests-jl/56987 "2021-03-12T00:48:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![bdoolittle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdoolittle/32/17053_2.png) [@bdoolittle](https://discourse.julialang.org/u/bdoolittle)\
**Post date:** [March 12, 2021, 12:48am UTC](https://discourse.julialang.org/t/how-to-improve-include-performance-in-runtests-jl/56987/1 "2021-03-12T00:48:11Z")

</div>

Hello, Julia users. I’ve recently noticed that when I run the unit tests of my module, a significant amount of time appears to be spent on include statements like `include("test/path/to/file.jl")` where the included files contains a `@testset`. For example, when I run `@time include("test/path/to/file.jl")` it tends to takes a few seconds longer to run than the contained testset which I time with `@time @testset begin .... end`. I realize that the overhead of `include()` is probably just something you have to accept when using Julia. However, I’m sure there are also some best practices to help reduce this overhead.

What are some tips and tricks to reduce the overhead of including tests in the `runtests.jl` file?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 12, 2021, 1:12am UTC](https://discourse.julialang.org/t/how-to-improve-include-performance-in-runtests-jl/56987/2 "2021-03-12T01:12:01Z")

</div>

Interesting. It is something related to the compilation time which is being measured, but there is indeed something odd. In one simple package of mine, I did the following:

I have a `@testset begin .. end` block. I copied exactly that block into another file, but left it in the `runtests.jl` file as well. And added time measures. Thus, my `runtests.jl` file looks like:

```julia
@time @testset "Tests" begin
    ...
end
@time include("./testset.jl")

```

and then I changed the order to:

```julia
@time include("./testset.jl")
@time @testset "Tests" begin
    ...
end

```

The results are:

1. With `@testset` before `include`:

```julia
Test Summary: | Pass Total
XLStats.jl | 30 30
  1.014827 seconds (2.13 M allocations: 107.128 MiB, 9.04% gc time) 
Test Summary: | Pass Total
XLStats.jl | 30 30
  0.053127 seconds (11.92 k allocations: 590.078 KiB) # include here

```

1. With `include` before `@testset`:

```julia
Test Summary: | Pass Total
XLStats.jl | 30 30
  2.177470 seconds (6.29 M allocations: 319.034 MiB, 1.76% gc time) # include here
Test Summary: | Pass Total
XLStats.jl | 30 30
  0.000577 seconds (4.70 k allocations: 140.172 KiB)

```

The overhead of `include` is therefore the difference between `0.053127` and `0.000577`, which excludes the compilation of the test set functions being called.

But why there is a much greater overhead when using include in the first run of the tests, I don’t know. Why something would take longer and allocate so much more to compile because of being included?

Edit: I don’t see any difference in similar tests when executing directly functions or including files that execute functions. That is something odd related to the test set.

---

<div class="post-metadata">

**Author:** ![bdoolittle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bdoolittle/32/17053_2.png) [@bdoolittle](https://discourse.julialang.org/u/bdoolittle)\
**Post date:** [March 12, 2021, 1:39am UTC](https://discourse.julialang.org/t/how-to-improve-include-performance-in-runtests-jl/56987/3 "2021-03-12T01:39:14Z")

</div>

This is certainly what I’m seeing as well. Thank you for sharing the code examples!