Installing dependencies for separate Project.toml during package development/testing

Tooling Package Management
,
1

I’m developing a package A wich has some unit tests as well as some examples.
I want my unit tests to be fast, so I seperated the examples from my unit tests using an additional Project.toml (this saves me a few minutes on every CI run on the tests).

The structure of my package looks like this:

A.jl/
| -- examples/
|  | example1.jl
|  | Manifest.toml
|  | Project.toml
| -- src/
|   | A.jl
| -- test/
|  | runtests.jl
|  | Manifest.toml
|  | Project.toml
| Manifest.toml
| Project.toml

with

example1.jl

using A
# Do stuff with A

Now building and testing A works fine:

(@v1.6) pkg> activate .
(A) pkg> build
(A) pkg> test

But now I’m struggeling to installing all dependencies for example/ in a way that I can just run include("example/example1.jl") to run my example.

What I managed to get running is the following:

julia --project=. -e 'import Pkg; Pkg.activate("examples"); Pkg.instantiate(); include("examples/example1.jl")'

Is this a good way to install dependencies and then run a Julia file?

What exactly is done during Pkg.test()? I think the procedure should be equivalent to what I want to achieve for example/: Install additional packages from project.toml and then include a file.

2

Marginal note: you could avoid the activate part using the --project command-line switch:

julia --project=examples -e 'import Pkg; Pkg.instantiate(); include("examples/example1.jl")'

Apart from that, this looks more or less like what I do (but let’s see if others have better ideas to suggest!)

Note that this works fine if the project defined in examples/Project.toml knows about its “parent project” A. In your case, maybe you used Pkg.dev to add project A as a dependency of project examples?


I’m not sure about all the details, but Pkg.test() involves spawning a new julia process with additional command-line flags (like --check-bounds=yes) in order to make sure your tests run in a clean and “hardened” environment so that you can catch as many errors as possible. You might not want all of that when running your examples.

Also, Pkg.test() somehow merges the main environment (defined by project A) with the testing environment (defined by test/Project.toml). This makes it possible for your tests to refer to project A without explicitly Pkg.developing it in the testing environment. I might not be up to speed on this topic, but last time I checked there were discussions about allowing such super-project / sub-project relations in broader contexts than just tests.

2 Likes
3

That is what I did wrong.

I tried to add A to examples/Project.toml with Pkg.add(".."). I thought that I need to add a path relative to examples/. In the end it happend to be an absolute path in my Manifest.toml (I tried so much stuff…), which worked on my machine but breaks everywhere else.

Of course my working directory is still A.jl/.
So for my case I need to add A with:

(examples) pkg> add .

The examples/Manifest.toml has now the correct repo-url = "..".

Thanks!

4

I think you’d be better off using Pkg.dev instead of Pkg.add:

(examples) pkg> dev .

That way, when running your examples, you’ll always use the latest version of A: as it stands in your filesystem, as opposed to the version that was fetched at the time of Pkg.add(".").