# Run test subsets with \`Pkg.test\`

**URL:** <https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050>\
**Category:** General Usage\
**Tags:** question\
**Created:** [November 13, 2017, 8:05pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050 "2017-11-13T20:05:28Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [November 13, 2017, 8:05pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/1 "2017-11-13T20:05:28Z")

</div>

It doesn’t seem to be possible to define groups of tests and then run only a given group of tests with `Pkg.test`. Is there an alternative recommended mechanism to achieve this goal?

Here is the use-case. I have a large package where running all the tests takes a long time. I am only working on a small part of it, so I would like to only run the tests related to the part that I am working on.

I could define test functions within the package, then import the package and call the test directly. This way I can organize things how I want, but I wonder if there is a more canonical way of doing it.

---

<div class="post-metadata">

**Author:** ![ggggggggg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ggggggggg/32/265_2.png) [@ggggggggg](https://discourse.julialang.org/u/ggggggggg)\
**Post date:** [November 13, 2017, 8:28pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/2 "2017-11-13T20:28:04Z")

</div>

I think this would be a great feature. I usually separate out my tests into smaller file, and `include` them all into `runtests.jl`. It’s pretty easy to comment out the slow tests while developing.

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [November 13, 2017, 8:37pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/3 "2017-11-13T20:37:19Z")

</div>

> [@ggggggggg](#):
>
> It’s pretty easy to comment out the slow tests while developing.

Of course, this is a workaround. But this is not very robust. You might forget that you commented something, and I don’t think constantly changing a source file without need is good practice.

---

<div class="post-metadata">

**Author:** ![ggggggggg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ggggggggg/32/265_2.png) [@ggggggggg](https://discourse.julialang.org/u/ggggggggg)\
**Post date:** [November 13, 2017, 8:39pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/4 "2017-11-13T20:39:47Z")

</div>

I agree, I just wanted to share it in case it would help temporarily.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 14, 2017, 8:28am UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/5 "2017-11-14T08:28:07Z")

</div>

> [@e3c6](#):
>
> You might forget that you commented something

Ideally, it shows up when you review the code before committing.

---

<div class="post-metadata">

**Author:** ![Ralph\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ralph_smith/32/10344_2.png) [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)\
**Post date:** [November 14, 2017, 2:24pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/6 "2017-11-14T14:24:28Z")

</div>

You can easily structure `runtests.jl` to select particular test groups depending on the environment.  
Either write your own logic (following, e.g. [DataStructures](https://github.com/JuliaCollections/DataStructures.jl/blob/master/test/runtests.jl)), or use `@includetests` from [TestSetExtensions](https://github.com/ssfrr/TestSetExtensions.jl).

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [November 14, 2017, 2:28pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/7 "2017-11-14T14:28:54Z")

</div>

> [@Ralph\_Smith](#):
>
> …depending on the environment.

What do you mean by “environment”? Environment variables?

---

<div class="post-metadata">

**Author:** ![Ralph\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ralph_smith/32/10344_2.png) [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)\
**Post date:** [November 14, 2017, 2:30pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/8 "2017-11-14T14:30:15Z")

</div>

Both of the approaches I mentioned use `ARGS` which can be set interactively or on a command line, but other environment variables could also work.

---

<div class="post-metadata">

**Author:** ![ettersi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ettersi/32/6829_2.png) [@ettersi](https://discourse.julialang.org/u/ettersi)\
**Post date:** [May 7, 2018, 6:24pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/9 "2018-05-07T18:24:59Z")

</div>

+1 for an “official” way to do this!

---

<div class="post-metadata">

**Author:** ![xgdgsc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xgdgsc/32/608_2.png) [@xgdgsc](https://discourse.julialang.org/u/xgdgsc)\
**Post date:** [December 21, 2019, 4:05am UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/10 "2019-12-21T04:05:59Z")

</div>

Is there any update on this? Most languages I used supports running single test or groups. It is really convenient and productive to be able to integrate with modern editor like vscode to click a button above the test code block to run single tests or save multiple run configurations of tests. Which is the repo to open a feature request for this, julia or Pkg?

---

<div class="post-metadata">

**Author:** ![wookyoung](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wookyoung/32/157_2.png) [@wookyoung](https://discourse.julialang.org/u/wookyoung)\
**Post date:** [December 21, 2019, 2:12pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/11 "2019-12-21T14:12:52Z")

</div>

you could run the test files in a specific directory and path `using Jive`  
visit [https://github.com/wookay/Jive.jl](https://github.com/wookay/Jive.jl)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 21, 2019, 3:32pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/12 "2019-12-21T15:32:03Z")

</div>

I think that this is implemented in

> **[GitHub - ssfrr/TestSetExtensions.jl: Extensions to Julia's Base.Test](https://github.com/ssfrr/TestSetExtensions.jl)**
>
> Extensions to Julia's Base.Test. Contribute to ssfrr/TestSetExtensions.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Samuel\_Ainsworth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuel_ainsworth/32/15644_2.png) [@Samuel\_Ainsworth](https://discourse.julialang.org/u/Samuel_Ainsworth)\
**Post date:** [September 15, 2020, 5:06am UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/13 "2020-09-15T05:06:25Z")

</div>

Could we pretty please get this in Pkg itself? It’s super handy to be able to just run one test when developing the test or the code being tested.

---

<div class="post-metadata">

**Author:** ![thofma](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thofma/32/1691_2.png) [@thofma](https://discourse.julialang.org/u/thofma)\
**Post date:** [September 15, 2020, 9:41am UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/14 "2020-09-15T09:41:58Z")

</div>

In the latest Pkg version there is [https://github.com/JuliaLang/Pkg.jl/pull/1226](https://github.com/JuliaLang/Pkg.jl/pull/1226). This can be used to customize `Pkg.test(...)`. I am not sure if this landed in julia 1.5.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 31, 2020, 5:10pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/15 "2020-10-31T17:10:01Z")

</div>

Has that landed in 1.5? Is the final form documented somewhere? I tried to follow that thread, but I could not find out how to do it at the end.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 31, 2020, 6:52pm UTC](https://discourse.julialang.org/t/run-test-subsets-with-pkg-test/7050/16 "2020-10-31T18:52:52Z")

</div>

It allows Pkg to pass arguments to the test process, but Pkg itself is not a testing framework so it is up to the testing framework you use to make use of those arguments.