Accesing a Registry on Private Github


#1

I’ve just made myself a private registry for my work packages, following the nice instructions here. My organisation uses private github for most of our code, so I’ve put it there. When I try to do a package update, Pkg tells me it “failed to fetch from repo”. But I can successfully fetch if I navigate to the directory in ~/.julia/registries and do git fetch.
Is Pkg not using my ssh credentials? Should I expect this to work?

From looking at the discussions around this topic, it seems like a user guide would be extremely useful for private registries, and some common edge cases. I would volunteer to do all the requisite learning and writing except I’m already at least two levels deep into “I probably shouldn’t spend time on this but it’s interesting” work.


#2

AFAIK Pkg uses HTTPS access, and for Github/ssh you want the git@... protocol. Just change the remote for origin.

https://help.github.com/articles/changing-a-remote-s-url/


#3

I see. That’s going to a problem, as our security set up doesn’t allow for anonymous access to our private repos. I guess I should raise an issue against Pkg.


#4

I don’t see what “anonymous” means in this context. How do you usually authenticate to Github for your own repos (regardless of Julia)?


#5

We authenticate git commands using ssh keys only. This appears to be what happens when you insist on 2fa for all accounts in your organisation. https always fails, even if you enter username and password correctly. And Pkg never prompts for username and password.


#6

If you use ssh keys, why should it need a password?

In any case, I have private repos on Github working fine with Pkg, but perhaps organizational setups are different. My ~/.gitconfig has

[github]
        user = my_github_username
[credential]
        helper = store

#7

I had read this to mean “Pkg always uses HTTPS access”, implying that ssh wasn’t possible. That’s why I was expecting Pkg to prompt for a username/password.

Your suggestion to use helper = store works for when I’m off-network, but not when I’m on-network. It seems that Pkg isn’t using my ssh config. Whilst I’m on the network, I am able to fetch the registry from the command line whilst in ~/.julia/registries/my-registry/
Do you know if that’s the expected behaviour?


#8

I am an not an expert on Pkg internals, but first I would make sure everything works as it should just using the git command line.


#9

OK, thanks Tamas. Seems like git is working just fine. Hopefully someone who knows more about Pkg and Registries will have a look!


#10

Whats the remote origin url of the registry? Is it https or git@? Have you tried Pkg.setprotocol!("ssh")?


#11

The remote origin, using git remote -v is git@. However, in Pkg.Registry.status() it’s https.

Pkg.setprotocol! doesn’t seem to make a difference to the result of Pkg.update().


#12

So, I went digging around in the source for Pkg, and followed things all the way down into the libgit2 C library. At this stage, I couldn’t figure out which calls did what, and ran out of time to investigate further.

A bit more detail about the on-network case which is the problem: We use a socks proxy for ssh traffic leaving our network. This is wrapped in a call to netcat, specified in one’s ssh config file. It seems that libgit2, and be extension Pkg, does not look at ssh config by default. A StackOverflow answer, here, says:

Reading config settings from your OpenSSH config file at ~/.ssh/config isn’t supported by libgit2 because it isn’t support by libssh2. If you want to read settings from there, you have to do it yourself.

If this is accurate, then someone will need to add support for ssh config files into Pkg, or maybe LibGit2.jl.

I’d be happy to hear any more ideas about how to achieve this, but it seems likely that the answer is to raise an issue to request this feature, and probably implement it myself if I want it quickly.


#13

Is this earlier discussion helpful? How to specify the SSH key for Pkg to use


#14

In addition to the SSH key, I need to read a ProxyCommand from the ssh config. So that earlier discussion doesn’t quite do enough.

Your suggestion prompted me to look up the specific issue of doing ProxyCommand-type things when using programs which use libssh2. I haven’t found a solution yet, but it might be useful.