Run Pkg.test with several processors

I have tests that use several processors. If I run julia -p 4 and then Pkg.test I only get 1 as a worker.

2 Likes

For now, you have to handroll your parallellization (see https://github.com/JuliaStats/Distributions.jl/blob/master/test/runtests.jl). There’s been some work towards exposing the test infrastructure that Base uses to run the tests to packages, but it isn’t ready yet (and might not happen).

1 Like

Looking at either the link you provide or my CPU load, it does not seem that the tests run in parallel?

Sorry, what is your question?

If the link is correct? I guess the link is supposed to point to a test suite that runs in parallel? Was there maybe an old version of Distributions, with parallel test suite?

You can have a look at our solver:
https://github.com/lanl-ansi/Juniper.jl

Unfortunately currently only properly working in julia v0.6 but that part should be the same

Yeah, it was removed in no parallel · JuliaStats/Distributions.jl@84d6b39 · GitHub. I should have linked to the file at a specific commmit.

1 Like

Do you know why it was removed? Is it fragile, does it produce bad error messages? I experimented with similar things as in Distributions.jl, but it was always fragile.

1 Like

Okay thanks. I am looking for something different however. Very much like the situation in Distributions.jl. Run many include("test_foo.jl") in parallel.

As of Julia v1.3 , is there any example on running test sets in parallel, maybe with multi-threading?

4 Likes

On my machine, running

cd("$your_pkg_path")
c:\bin\Julia-1.3.1\bin\julia -p auto -e "include(raw\"test\runtests.jl\")"

leads to those ressources usage

BEFORE EXE

and

DURING EXE

It seems test run on all my available cores.
That’s left to tell how we can repro it inside the atom IDE…

Following on from @felipenoris 's question, is there an update on this for Julia 1.6 or 1.7?

1 Like

It would be nice to have an option to run tests in parallel with interface similar to how precompilation is done from 1.6 onwards.

2 Likes

Strong agree, would speed up my iteration a lot.

1 Like

Same here

1 Like

Any usable solution now?

ReTest.jl or TestItemRunner.jl are two working solutions

2 Likes