Private repos and private registry on Azure DevOps

Tooling Package Management
, ,
1

Hi,

I’m interested in setting up a private registry on Azure DevOps. I already have several private package there. As a start, I tried to simply

pkg> add https://MyOrg@dev.azure.com/MyOrg/MyProject/_git/MyPackage.jl

“pkg> add” doesn’t seem to like the @ after MyOrg, so I tried

julia> using Pkg; Pkg.add(PackageSpec(url="https://MyOrg@dev.azure.com/MyOrg/MyProject/_git/MyPackage.jl"))
    Cloning git-repo `https://MyOrg@dev.azure.com/MyOrg/MyProject/_git/MyPackage.jl`

This prompts me for credentials, but I am confused because I can

$ git clone https://MyOrg@dev.azure.com/MyOrg/MyProject/_git/MyPackage.jl

without credentials. However, even after entering my credentials, “Pkg.add” still fails.

I’m pretty clueless about SSH stuff, but I’m guessing it is related. My command line git seems to be finding my SSH stuff, but Pkg isn’t. How can I fix that?

When I asked on Slack, someone responded:

We use Azure at my job. On Linux, I can only get it to work with SSH, where the “@“ isn’t misunderstood. I haven’t been able to get it to work with plain HTTPS authentication either, it needs a PAT for authentication. It’s a bit annoying, if you use Azure Pipelines, because the pipeline only works with HTTPS. But you can make it work by using git URL rewrites.

That :point_up:has me a little concerned because the point of me trying to set this up is so that I could use Azure Pipelines and setting up a pipeline with URL rewrites doesn’t sound like fun (whatever that is).

Any ideas?

If anyone else has experience with Julia and Azure DevOps, I’d appreciate any help. Thank you :pray:

2

Hi @anon67531922 this issue of command line git working OK and Julia failing has come up in the past. In the context of using proxies. The issue is/was the libgit2 library which Julia uses for git fetches.
I thought the issue had been solved a long time ago - it really was not a Julia issue but an issue with that library.
Have a look at this issue: https://github.com/JuliaLang/julia/issues/30635

3

Hello @anon67531922 Which OS are you running Julia on?

4

Here is what I have understood about Azure: If you use HTTPS, it will prompt you for credentials, but it will fail. This is a security setting in Azure. You have to use PAT or SSH authentication. The PAT authentication will open a browser window and then take you back or something like that. I have never used it, but some of my coworkers use it. So, to the best of my knowledge, the fact that it fails isn’t related to the package manager, but Azure’s security settings. I would recommend setting it up with SSH. I have found it to be the simplest way to work with repos.

I have run into an issue, that I think is related to the package manager, though. The problem arises when you try to use Azure Pipelines with a private registry. The pipelines only seem to work with PAT authentication, which must either be passed to git or put in the URL. Because the URLs for the packages in the registry are hardcoded, the only way I could get it to work was using a git URL rewrite (git config --add url."https://pat:$(System.AccessToken)@...".insteadOf ...). The package manager, however, ignores local git configurations, which means that you must configure this using git config --global. This leads to some issues when using local build agents, if they are not configured to wipe the home directory after each run, since the configuration will persist and leave an invalid access token in the configuration.

It isn’t impossible to work around, though. Just put an export HOME=$BUILD_SOURCESDIRECTORY at the top of every script in the pipeline and set the global git configuration at the top, before checking out your packages. It adds a few lines of boilerplate, but totally possible to live with.

1 Like
5

Hi @johnh :wave:

I appreciate your help. I am on Windows, which I think makes matters worse :sweat_smile:

6

Thanks @Evey :raised_hands:

That seems a little too complicated for my devops skills :sweat_smile:

The way I did this before was with gitsubmodules. That actually worked fine initially despite several warnings from people who know more about this stuff than I do. As things grew, I wish I had heeded those warnings, so I’m trying to move away from gitsubmodules (which required me to check in my Manifest with relative paths for private repos - which is fine for me actually).

Now, I am considering a couple options:

  1. Mono repo (all privates packages globbed together in one repo)
  2. Use the Azure Pipelines checkout feature to pull in the private packages and then build the image

Both options above require me to check in my Manifest with relative paths, e.g.

pkg> dev ../../PrivatePackage1

I made some headway with Option 2. I was able to checkout my private Julia packages and build the image, but then it failed on the deploy step. It couldn’t find my (Kubernetes) manifest files. I have a ticket open with Azure about that and have a few things to try (they suggest checkout in both the build and deploy step so it can get the Agent directory correct :thinking:)

I haven’t tried v1.5 yet, but I understand it has better support for subdirectories of a repo, so the mono repo idea doesn’t sound so crazy to me. Any opinions on that?

PS: I am actually grateful to Microsoft for being so careful about security so I don’t see this as a Microsoft problem.

7

@anon67531922 Does https://MyOrg.visualstudio.com/MyProject/_git/MyPackage.jl give you the same repo? It does for my organisation.

Also, I’m sorry if I made it sound complicated with the pipeline. If you only use the Azure build agents, the home folder should be recreated every time, as far as I understand. In that case, you simply need to put:

git config --global --add url."https://pat:$(System.AccessToken)@<your.git.repo>".insteadOf https://<your.git.repo>

before the part of the pipeline where you add your packages, where you replace <your.git.repo> by the MyOrg.visualstudio.com/... URL I suggested above. I can only recommend it. :blush:

1 Like
8

Thank @Evey :raised_hands:

Does https://MyOrg.visualstudio.com/MyProject/_git/MyPackage.jl give you the same repo? It does for my organisation.

Yes. I didn’t realize that :+1:

I’m a little lost. Are you suggesting that if I do this :point_up:, I should be able to run a private registry on Azure Repos? That would be pretty awesome :slight_smile:

You’ve already been super helpful. Is it too much (understandable) to ask if you could help walk through that a little bit? :pray::blush:

Naively, what I would do is put something like

RUN ["julia", "-e", "using Pkg; Pkg.Regsitry.add("https://MyOrg.visualstudio.com/MyProject/_git/MyRegistry")

in my Dockerfile, but I wouldn’t expect that to work because MyRegistry is a private Azure repo. If I understand, your git config suggestion in the pipeline would make this work in the Dockerfile?

9

That is what I am suggesting. :grin:

I’m not entirely sure which Dockerfile you’re referring to. I assume you’re building a Docker image with your packages inside? Unless you want to put a secret in there, like an SSH key that has access to your repos, I would advise against actually adding the registry inside the Docker. But the pipeline is given a temporary Personal Access Token (PAT), that gives it read access to all your repos inside the project that it is run in. As long as git is configured inside the pipeline YAML script to use the PAT to authenticate when retrieving your packages, then a private registry works. To set it up, you need to put the git config command I suggested above in a script in the YAML file somewhere before you run julia. Otherwise the package manager won’t authenticate properly.

1 Like