Having local packages in a monorepo depend on one another

Tooling Package Management
, ,
1

We’re using a monorepo to maintain our julia packages, with a directory structure along the lines of:

julia
    PackageOne
    PackageTwo
    PackageThree

I can add these packages to my top-level Project by running Pkg.develop(path="<path_to_package>") and can subsequently use each package.

I am having trouble having a package depend on another. I’ve tried various versions of Pkg.add, but they all seem to expect each package individually to be a git repo.

With, say, PackageThree activated, I’ve tried:

julia> Pkg.add("PackageOne")
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
ERROR: expected package `PackageOne [8b4b255c]` to be registered

julia> Pkg.add(path ="<path_to_package_one>")
ERROR: Did not find a git repository at `<path_to_package_one>`

Seeing that I can develop packages at paths, is there a way to also depend on other packages at paths?

3 Likes
2

This is what you want do do again for the local projects too.

1 Like
3

Sorry, I think I am misunderstanding something. The packages are already in my Project.toml in .julia/environments/1.5.

Adding the package / developing when I check out my local package doesn’t work either (run from ~/myrepo/julia/):

(@v1.5) pkg> activate PackageThree
 Activating environment at `~/myrepo/julia/PackageThree/Project.toml`

julia> using Pkg

julia> Pkg.develop(path="PackageOne")
Path `PackageOne` exists and looks like the correct package. Using existing path.
  Resolving package versions...
No Changes to `~/myrepo/julia/PackageOne/Project.toml`
No Changes to `~/myrepo/julia/PackageOne/Manifest.toml`

julia> Pkg.add("PackageOne")
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
ERROR: expected package `PackageOne [8b4b255c]` to be registered

julia> Pkg.add(path="PackageOne")
ERROR: Did not find a git repository at `PackageOne`
4

Just like you did Pkg.develop to add the packages to your global environment (~/.julia/environments/1.5) you should use Pkg.develop when you add them as dependencies of each other. Here is an example:

Generate two local packages:

$ jlpkg generate A; jlpkg generate B
 Generating  project A ...
 Generating  project B ...

Add them with develop and the path:

$ jlpkg --project=. dev A B # Add both to "global env"
  Resolving package versions...
Updating `/tmp/Project.toml`
  [a8860211] + A v0.1.0 `A`
  [c725e7a7] + B v0.1.0 `B`
Updating `/tmp/Manifest.toml`
  [a8860211] + A v0.1.0 `A`
  [c725e7a7] + B v0.1.0 `B`

Manifest looks good:

$ cat Manifest.toml 
# This file is machine-generated - editing it directly is not advised

[[A]]
path = "A"
uuid = "a8860211-dfc0-462f-9fbf-c3432d7e395f"
version = "0.1.0"

[[B]]
path = "B"
uuid = "c725e7a7-9ad5-4b2e-b018-0ca07287f03e"
version = "0.1.0"

Add B as a dependency to the A project:

$ jlpkg --project=A dev B # Add B as a dependency to A
  Resolving package versions...
Updating `/tmp/A/Project.toml`
  [c725e7a7] + B v0.1.0 `../B`
Updating `/tmp/A/Manifest.toml`
  [c725e7a7] + B v0.1.0 `../B`

Sync the global manifest to update A’s dependencies

$ jlpkg --project=. resolve # Sync the global manifest
[...]

Make sure A properly depends on B:

$ cat Manifest.toml 
# This file is machine-generated - editing it directly is not advised

[[A]]
deps = ["B"]
path = "A"
uuid = "a8860211-dfc0-462f-9fbf-c3432d7e395f"
version = "0.1.0"

[[B]]
path = "B"
uuid = "c725e7a7-9ad5-4b2e-b018-0ca07287f03e"
version = "0.1.0"
3 Likes
5

add is not the right tool in this scenario, but if it were the incantation would be Pkg.add(path=<path_to_julia>, subdir="PackageOne").

Myself I would just make a registry and register the packages, then go on as usual and not worry about relative paths and stuff.

1 Like
6

Registries seem to have to be repos too:

(@v1.5) pkg> registry add .
    Copying registry from `~/myrepo/julia/MyRegistry`
ERROR: no `repo` entry in `Registry.toml`.

I took a look at 7. Registries · Pkg.jl but it only adds via url.

7

I don’t think its necessary to use a registry for this. The previous advice about adding the packages one by one is best.

1 Like
8

Aha! What I was missing is that calling Pkg.develop(path="julia/PackageOne") once PackageThree is activated adds it to the Project.toml, so Pkg.add is not needed. Thank you.

1 Like
9

Agree, the monorepo approach seems at odds with using a registry: if all the packages are in a single repo and they get updated in lockstep, what is the purpose of a registry?

1 Like
10

Not really. That error message only says that Registry.toml must contain a repo field, not that it’s actually going to do anything with it. Although it certainly would be most convenient and idiomatic to have the registry in a separate repository, the rest of Pkg, outside of repository add, wouldn’t know or care where a repository came from, say a symbolic link from ~/.registries/MyRegistry to <path to monorepo>/MyRegistry.

It depends on how the packages are actually used or deployed but primarily it gives you the option of falling out of lockstep and allows you to use Pkg in a standard way.

1 Like
11

They don’t, we should just remove the check that this field exists, imo.

1 Like