I have a repo with a package and a subpackage, and I wanna register both. I suppose it will work just fine, but is there anything I should be careful about? Feedback from people who manage monorepos?
One small gotcha about registration is if you are using the commentbot to register, you need to make a separate comment invoking @JuliaRegistrator for each package you want to register, rather than sticking multiple commands into a single comment.
There’s also some trickiness in effectively developing in such a setup. For example, you might want to co-develop some features by Pkg.deving one package into another. But at release time, they are released independently and users won’t get the pair of packages together (unless you are careful with compat bounds to get exactly what you want). So even if you dev them for some time during development, you probably want to test them independently. A good strategy can be to have multiple CI workflows, one for each package independently (i.e. only Pkg.deving a single package, using the release version of the other), as well as one testing them together (Pkg.deving both packages), so that you can have all the information at PR-merge time about what works and what doesn’t. Arrow.jl does this for example: arrow-julia/.github/workflows/ci.yml at main · apache/arrow-julia · GitHub.
For example, if your dev version of PkgA depends on a feature in your dev version of PkgB, a good outcome to see when merging that PR is that the PkgA-only workflow failed due to the failure to find the correct version of PkgB in the registry (since you set compat bounds correctly), PkgB’s tests alone succeeded, and the joint tests succeeded.
However, a better way to do that could be to jointly develop on a branch, then once you are happy the feature in PkgB meets your needs in PkgA, split off the PkgB feature to a new PR (with no code changes to PkgA), merge & tag PkgB, then rebase your dev PR on main (to drop off the changes to PkgB), use the new feature in PkgA, bump compat, merge and tag again.
All this to say, it is a more complicated development scenario that can involve more git or more careful reading of CI to ensure both packages work independently and work together.
5 Likes
One thing that may or may not be obvious is that if you happen to use the layout
Package
├── Project.toml
├── src
│ └── Package.jl
└── SubPackage
├── Project.toml
└── src
└── SubPackage.jl
then Package will be distributed with an unused copy of the SubPackage code.
I would recommend having all packages in sibling directories, regardless whether some is main and some is sub. If you already do, we’re on the same page.
5 Likes
And how does it work for tagging on GitHub? Does it become useless with a monorepo?
TagBot doesn’t work (it has the feature but it is broken / never worked), but you can push tags manually (Registrator.jl will tell you the syntax when you register), usually of the form PkgB-v0.1.0 etc. It is not useless, since it is handy to see the versions of the subpackages, and Documenter can be configured to built stable docs for each subpackage on tags of that form (see https://documenter.juliadocs.org/stable/man/hosting/#Deploying-from-a-monorepo).
1 Like
What about the GitHub actions that are typically used in the CI files?
I’m assuming stuff like
- uses: julia-actions/cache@v1
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-runtest@v1
- uses: julia-actions/julia-processcoverage@v1
will not necessarily work if the package in question is not at the root?
It turns out the Arrow.jl package CI files are a good reference for that, thanks!
https://github.com/apache/arrow-julia/blob/main/.github/workflows/ci.yml
For future reference, these actions only need a project specification:
- uses: julia-actions/julia-runtest@v1
with:
project: ${{ matrix.pkg.dir }}
As for code coverage with Codecov, it is cumulative, so we don’t need to store the coverage information in separate files
1 Like