Run test subsets with `Pkg.test`

question

#1

It doesn’t seem to be possible to define groups of tests and then run only a given group of tests with Pkg.test. Is there an alternative recommended mechanism to achieve this goal?

Here is the use-case. I have a large package where running all the tests takes a long time. I am only working on a small part of it, so I would like to only run the tests related to the part that I am working on.

I could define test functions within the package, then import the package and call the test directly. This way I can organize things how I want, but I wonder if there is a more canonical way of doing it.


#2

I think this would be a great feature. I usually separate out my tests into smaller file, and include them all into runtests.jl. It’s pretty easy to comment out the slow tests while developing.


#3

Of course, this is a workaround. But this is not very robust. You might forget that you commented something, and I don’t think constantly changing a source file without need is good practice.


#4

I agree, I just wanted to share it in case it would help temporarily.


#5

Ideally, it shows up when you review the code before committing.


#6

You can easily structure runtests.jl to select particular test groups depending on the environment.
Either write your own logic (following, e.g. DataStructures), or use @includetests from TestSetExtensions.


#7

What do you mean by “environment”? Environment variables?


#8

Both of the approaches I mentioned use ARGS which can be set interactively or on a command line, but other environment variables could also work.


#9

+1 for an “official” way to do this!