# Standard ways to run common commands
**URL:** https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154
**Category:** Internals & Design
**Created:** [November 7, 2016, 6:38am UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154 "2016-11-07T06:38:00Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@djsegal](https://discourse.julialang.org/u/djsegal)
#### Post date: [November 7, 2016, 6:38am UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/1 "2016-11-07T06:38:00Z")
Over the course of developing a project, there are certain commands that get executed all the time.
These include running:
- Tests
- Subsets of tests
- The application’s main function
How can we introduce standard ways to perform these functions?
* * *
Also, a standard way to run functions would give:
- people a nice list of hooks to plug their initialization code.
- a way to warn people that their packages don’t meet REQUIRE specifications and other stuff
---
### Author:  [@vchuravy](https://discourse.julialang.org/u/vchuravy)
#### Post date: [November 7, 2016, 7:22am UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/2 "2016-11-07T07:22:01Z")
Are you looking for something more that `Pkg.test("Package Name")` for running tests? Running subsets of tests might be more difficult , because there is no default testing framework people are supposed to use. (And that seems to be a good idea, otherwise we wouldn’t necessarily had `FactCheck` → `BaseTestNext`).
Do you have an example for other languages, that have something like this? It seems to me that this is mostly a problem that is solved by frameworks in other languages (`Rails` vs `Ruby`).
---
### Author:  [@musm](https://discourse.julialang.org/u/musm)
#### Post date: [November 7, 2016, 12:11pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/3 "2016-11-07T12:11:45Z")
> [@djsegal](#):
>
> Subsets of tests
+1 It’s a real pain to have to comment out the tests you want to run individually when developing a package.
Something like `Pkg.test("MyModule","TestSet1")` would be fantastic.
---
### Author:  [@tshort](https://discourse.julialang.org/u/tshort)
#### Post date: [November 7, 2016, 9:13pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/4 "2016-11-07T21:13:56Z")
For many automation tasks, you can just write a function that does what you want. I wrote the following package to help with automating routine activities:
> **[GitHub - tshort/Maker.jl: A make-like system for data analysis](https://github.com/tshort/Maker.jl)**
>
> A make-like system for data analysis. Contribute to tshort/Maker.jl development by creating an account on GitHub.
It’s like the make utility in that code runs based on updated dependencies. It’s focused more on analysis activities rather than file-based manipulations.
---
### Author:  [@ssfrr](https://discourse.julialang.org/u/ssfrr)
#### Post date: [November 10, 2016, 1:15am UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/5 "2016-11-10T01:15:17Z")
Also check out my new TestSetExtensions package ([https://github.com/ssfrr/TestSetExtensions.jl](https://github.com/ssfrr/TestSetExtensions.jl)). It has a way to selectively run different test files without assuming you’re writing your tests with anything except the basic Base.Test stuff from 0.5 (or BaseTestNext on 0.4).
\
-s
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [November 10, 2016, 4:23pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/6 "2016-11-10T16:23:19Z")
I think the next steps here are to converge how tests are run for packages and Base Julia. @kslimes has done a lot of work towards this with a series of epic changes to make Julia’s base tests use the new test stuff (originally written by Iain Dunning). Base runs lots of tests in parallel; should packages automatically run their various test files in parallel? What are some of the other differences at this point?
---
### Author:  [@djsegal](https://discourse.julialang.org/u/djsegal)
#### Post date: [December 8, 2016, 11:18pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/8 "2016-12-08T23:18:27Z")
Hi Stefan,
Can you elaborate on this a bit? What files give the flavor of what you’re talking about most? Maybe, show:
- a few good examples from Juila’s base tests
- some repos that you think reinvented the wheel too much (and highlight the need for standardization)
---
### Author:  [@kslimes](https://discourse.julialang.org/u/kslimes)
#### Post date: [December 9, 2016, 5:24pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/9 "2016-12-09T17:24:12Z")
Hello @djsegal,
I think base Julia’s `test/math.jl` and `test/libgit2.jl` give a good example of using test sets. `test/runtests.jl` “farms out” the test files to parallel workers and gathers some performance information along with the results (it’s also a cool example of using `@async`, I think).
Does that answer your first question?
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [December 9, 2016, 6:05pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/10 "2016-12-09T18:05:12Z")
Currently most packages don’t have enough tests that they need the kind of parallel testing functionality that Base Julia has, but some probably do already and more and more will – we might as well make it a built-in standard and use that in Base as well as packages. That’s what’s being suggested. (So what Katie said, in other words.)
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [December 9, 2016, 6:18pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/11 "2016-12-09T18:18:41Z")
> [@kslimes](#):
>
> I think base Julia’s test/math.jl and test/libgit2.jl give a good example of using test sets. test/runtests.jl “farms out” the test files to parallel workers and gathers some performance information along with the results (it’s also a cool example of using @async, I think).
Oh, I didn’t know you got multiple workers on Travis! I already have everything divided into testsets, so I assume I can just `@spawn` each testset expression (mine are all independent)? How do you get the number of processes to make on Travis (NUM\_CORES)?
Here’s a snippet of what it looks like:
>
Since every testset is independent, they could all run at the same time.
Showing how to do parallel testing on Travis would be a really nice blog post that could really help development times!
---
### Author:  [@djsegal](https://discourse.julialang.org/u/djsegal)
#### Post date: [April 28, 2017, 9:04pm UTC](https://discourse.julialang.org/t/standard-ways-to-run-common-commands/154/12 "2017-04-28T21:04:34Z")
Are there plans to make Julia’s test infrastructure closer in feel to other languages?
Maybe @testset would benefit from some flags, including:
- `focus: true` // only run this test
- `skip: true` // skip this test
- `speed: slow` // allow skipping these tests
- `type: solver` // similar to `speed` but w/ file type
_// note: these were just picked up off a quick scrub of [rspec’s docs](https://www.relishapp.com/rspec/rspec-core/v/2-4/docs/command-line/tag-option)_
* * *
**edit:** maybe allow timeout setting also ([see this post](https://discourse.julialang.org/t/is-there-way-to-make-testsets-fail-after-n-seconds/3424))