Private registry

General Usage
1

Hi,

I would like to be able to run CI/CD using GitHub Actions in private packages.

Let’s say I have two private packages, PkgA and PkgB, with PkgB depending on PkgA.

I would like to be able to set up a private registry so when I run the ci.yml file in PkgB as a GitHub action, it does not fail saying something like: ERROR: Build path for PkgA does not exist:

Has anyone built a private registry and would like to provide the steps to cope with a situation like this one?

Thanks you!

2

For a small number of private repos, it’s probably easier just to skip the registry and refer to the package dependencies by git ssh URLs (git@github.com:user/Package.jl). You’ll need a manifest for your CI run to make sure those URLs are downloaded, or use Pkg.add(url=..., rev=...) as part of a CI setup step. As part of the setup you’ll also need to make sure your CI has ssh credentials to download the private packages.

3 Likes
3

I think people usually use LocalRegistry.jl for these kinds of purposes once there are too many packages to do it manually. You’ll have to make that registry available to the CI jobs and make sure only CI and you can authenticate themselves to it, which may be a little challenging (unless your private packages are private repos on github).

6 Likes
4

Seconding the LocalRegistry.jl recommendation. I use a private registry for all my packages, and LocalRegistry.jl makes it effortless to maintain. Just call its register function from the REPL, and the new package/version is registered and immediately available for addìng. That’s even less friction compared to using General, and there are no waiting periods - especially important for new versions.
Of course, the registry repo need to be accessible to CI, which means creating a token in the case of GitHub private repos.

4 Likes
5

great answers everybody!

I will check out LocalRegistry.jl.

If anyone considers that it is worth sharing a code snippet for any step of this kind of setup, please do share it.

Thanks!!

6

But how many tokens do you need? Note that GitHub deploy keys can grant access to a single repository only. If PkgA depends on PkgB, which in turn depends on PkgC, it seems that with this approach, PkgA would need a token to access PkgC repo (assuming they are all private). This does not scale well and I think will get quickly out of hand even for a small number of packages, since intermediate dependencies can vary with time.

Is there a better way?

1 Like
7

I personally went with the simple solution of creating a separate github SSH key for my account and using it in CI of my packages-that-depend-on-other-my-package. This key lets CI clone any of my repos and doesn’t require any complex setup - CI just does the same as I do on my computer to clone them. Don’t see any security issues with this as long as I control CI scripts that run with these keys.

Not saying that this solution is always optimal, of course. I’m not familiar with per-repo keys that you discuss at all.

1 Like
8

I see. So you give your account key to each of your private repos that needs to use other private repos as a secret?

9

Exactly!

10

Coming back to this. If I understand correctly your account key is gives read and write access to all your repos, so this seems a bit insecure.

Also, you have to manually add your key to all repos, correct? There is no way to have one secret be accessible to all your repos?

11

I add the same key to all repos that need it, yes.
With this command:

cat your_julia_id_rsa_file | gh secret set SSH_PRIVATE_KEY

And use this step on CI:

      - run: |
            mkdir -p ~/.ssh
            ssh-keyscan github.com >> ~/.ssh/known_hosts
            ssh-agent -a /tmp/ssh_agent.sock > /dev/null
            ssh-add - <<< "${{ secrets.SSH_PRIVATE_KEY }}"

Not sure if all commands are really required here, I found this snippet somewhere and it works.

2 Likes
12

Ok thanks for the snippet!