In our package, we have a number of dependencies used for testing only. For end users of our package, we don’t wish to enforce those dependencies. It seems that the suggested way of achieving this is using Test-specific dependencies.
The problem with that technique is that, as far as I can tell, it only works when running the test suite through Pkg.test(), by creating a new temporary environment. But in our package, we have various ways of testing our code: the standard test suite, a script that allows testing a subset of the code, more extensive benchmarks, plotting the output, etc. These currently live as independent scripts in the test/ directory, e.g. test/runbenchmark.jl.
Now we’re wondering what the best way is to have a separate set of dependencies for those scripts only? Ideally an independent set of dependencies per script (in particular, we don’t want heavy dependencies like plotting unless really needed). One way is to programmatically add dependencies in the scripts themselves, so for example runbenchmark.jl could start with:
The idea would be to have Pkg.test() in the top-level project run the default test suite only, since people are used to running this command to run the unit tests, and would expect this to work.
For more advanced test scripts, I’d be ok with a different syntax. But importantly, I’d like at least some of these scripts to have the exact same dependencies as the default test suite, without having to repeat these dependencies in several project files. (We have for example one script that runs a subset of the default test suite.) I don’t think your second suggestion allows that, since it’d use the dependencies defined in the top-level Project.toml file to run the default tests?
For your first suggestion, I still get Package Pkg not found in current path. It also doesn’t like my startup.jl. The below works, but doesn’t feel any less hacky than the solution I posted above:
run(`$(Base.julia_cmd()) --startup-file=no --project=$(@__DIR__) -e