Reduce compilation time in Testing

In the package I am (co-)developing, we write a lot of tests. Now we are running into the problem, that the tests run quite long (currently for example a little more than an hour).

In our output like

So most of these, also the following ones, spent their time on compilation – sure these are all specialised functions for the separate manifolds – one user would not notice this much, when using just one of them. Still – are there any good ideas on how to reduce compilation time here, maybe?

2 Likes

You are more experienced that me :wink: But from my modules the package SnoopCompile.jl · SnoopCompile really helped to reduce compile times.

I also suffer from similar problems, for many packages, I can reduce the time to run the tests by about 50-75% by running

julia --compile=min -O 1

It’s a bit tricky to use, since if you have some tests that are somewhat computationally expensive, those might become so slow with the above options that it takes longer time overall.

3 Likes

That sounds nice, I have to check how to add that to GitHub CI (after trying that locally).

The flags locally reduce computations for a few tests but increase them tremendously for others (from 30 seconds to 1000 seconds) so overall (locally on my machine) our tests run longer with those settings. But thanks for the idea.

I have not yet worked with SnoopCompile (and I am surely not experienced with recompilation) but I will take a look, thanks!

Update – I have a current variant, that splits my tests into two, runs some (previously determined) tests on the normal Julia and others on -O0 --compile=min, see

https://github.com/JuliaManifolds/Manifolds.jl/blob/kellertuer/faster-tests/.github/workflows/ci.yml

the one thing this does not yet seem to do is – upload the code coverage for the faster tests.