Private registry

Hi,

I would like to be able to run CI/CD using GitHub Actions in private packages.

Let’s say I have two private packages, PkgA and PkgB, with PkgB depending on PkgA.

I would like to be able to set up a private registry so when I run the ci.yml file in PkgB as a GitHub action, it does not fail saying something like: ERROR: Build path for PkgA does not exist:

Has anyone built a private registry and would like to provide the steps to cope with a situation like this one?

Thanks you!

For a small number of private repos, it’s probably easier just to skip the registry and refer to the package dependencies by git ssh URLs (git@github.com:user/Package.jl). You’ll need a manifest for your CI run to make sure those URLs are downloaded, or use Pkg.add(url=..., rev=...) as part of a CI setup step. As part of the setup you’ll also need to make sure your CI has ssh credentials to download the private packages.

2 Likes

I think people usually use LocalRegistry.jl for these kinds of purposes once there are too many packages to do it manually. You’ll have to make that registry available to the CI jobs and make sure only CI and you can authenticate themselves to it, which may be a little challenging (unless your private packages are private repos on github).

5 Likes

Seconding the LocalRegistry.jl recommendation. I use a private registry for all my packages, and LocalRegistry.jl makes it effortless to maintain. Just call its register function from the REPL, and the new package/version is registered and immediately available for addìng. That’s even less friction compared to using General, and there are no waiting periods - especially important for new versions.
Of course, the registry repo need to be accessible to CI, which means creating a token in the case of GitHub private repos.

3 Likes

great answers everybody!

I will check out LocalRegistry.jl.

If anyone considers that it is worth sharing a code snippet for any step of this kind of setup, please do share it.

Thanks!!

But how many tokens do you need? Note that GitHub deploy keys can grant access to a single repository only. If PkgA depends on PkgB, which in turn depends on PkgC, it seems that with this approach, PkgA would need a token to access PkgC repo (assuming they are all private). This does not scale well and I think will get quickly out of hand even for a small number of packages, since intermediate dependencies can vary with time.

Is there a better way?

1 Like

I personally went with the simple solution of creating a separate github SSH key for my account and using it in CI of my packages-that-depend-on-other-my-package. This key lets CI clone any of my repos and doesn’t require any complex setup - CI just does the same as I do on my computer to clone them. Don’t see any security issues with this as long as I control CI scripts that run with these keys.

Not saying that this solution is always optimal, of course. I’m not familiar with per-repo keys that you discuss at all.

1 Like

I see. So you give your account key to each of your private repos that needs to use other private repos as a secret?

Exactly!