Pkg.clone keeps asking for my SSH key

General Usage
, ,
1

Hello,

I am running Julia 0.7 on Ubuntu and when I try to clone a package, I get this problem:

julia> using Pkg

julia> Pkg.clone("git@github.com:eford/CORBITS.git")
┌ Warning: Pkg.clone is only kept for legacy CI script reasons, please use `add`
└ @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:445
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `git@github.com:eford/CORBITS.git`
Private key location for 'git@github.com': /home/duc299/.ssh/id_ed25519
Private key location for 'git@github.com' [/home/duc299/.ssh/id_ed25519]: /home/duc299/.ssh/id_ed25519
Private key location for 'git@github.com' [/home/duc299/.ssh/id_ed25519]: /home/duc299/.ssh/id_ed25519
ERROR: failed to clone from git@github.com:eford/CORBITS.git, error: GitError(Code:EAUTH, Class:Callback, Aborting, maximum number of prompts reached.)
Stacktrace:
 [1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Types.jl:120
 [2] #clone#2(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GitTools.jl:102
 [3] clone at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GitTools.jl:83 [inlined]
 [4] (::getfield(Pkg.Types, Symbol("##25#28")){Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1},String})(::LibGit2.CachedCredentials) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Types.jl:533
 [5] shred!(::getfield(Pkg.Types, Symbol("##25#28")){Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1},String}, ::LibGit2.CachedCredentials) at ./secretbuffer.jl:184
 [6] handle_repos_develop! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Types.jl:498 [inlined]
 [7] #add_or_develop#13(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:52
 [8] #add_or_develop at ./none:0 [inlined]
 [9] #develop#19 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:70 [inlined]
 [10] develop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:70 [inlined]
 [11] macro expansion at ./logging.jl:310 [inlined]
 [12] clone(::String, ::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:445 (repeats 2 times)
 [13] top-level scope at none:0

So it keeps asking me for my SSH key and it won’t listen to me when I tell it where the SSH key is. Also, regular git seems to work fine:

~ % ls -ld .ssh
drwx------ 2 duc299 duc299 146 Sep 19 01:10 .ssh/
~ % 
~ % ls -l .ssh/id_ed25519*
-rw------- 1 duc299 duc299 411 Sep 19 00:59 .ssh/id_ed25519
-rw-r--r-- 1 duc299 duc299  98 Sep 19 00:59 .ssh/id_ed25519.pub
~ % 
~ % cd tmp
~/tmp % 
~/tmp % git clone git@github.com:eford/CORBITS.git
Cloning into 'CORBITS'...
Warning: Permanently added the RSA host key for IP address '192.30.253.112' to the list of known hosts.
remote: Counting objects: 405, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 405 (delta 3), reused 13 (delta 3), pack-reused 389
Receiving objects: 100% (405/405), 85.34 KiB | 1.09 MiB/s, done.
Resolving deltas: 100% (210/210), done.

So the problem does not seem to be specifically with git. Also, the Pkg.clone line worked in two other computers. So I’m completely stuck. Why can’t Pkg.clone call git like on the command line? What’s missing?

2

Are you using a .ssh/config file to specify your git key?
If so LibGit2 doesn’t read those.
But command line git does.

Both command line git, and LibGit2 do pay attention to your ssh-agent, if you have one running.

I’ve also heard there is an environment variable you can set to tell julia the path.

3

No, I’m not using .ssh/config to specify the key. Git just seems to find it. It’s the only SSH key I have. And in any case, surely specifying a direct path to the key (like I did twice in the example I pasted) should tell LibGit2 where the key is.

4

The man page for ssh-agent says:

ssh-agent is usually started in the beginning of an X-session or a login session

I am currently connecting remotely to the computer that’s not working. Could that be the problem? Hmm…

local_computer % echo $SSH_AGENT_PID
1414
local_computer % ssh remote_computer
...
remote_computer % echo $SSH_AGENT_PID

remote_computer %

It looks like I might have been using ssh-agent all along. Do you think ssh-agent will be working if I go physically to the remote computer? (it’s running an X session right now).

5

Hmm… I do have ssh-agent running:

remote_computer ~/.ssh % ps x|grep ssh-agent       
16090 ?        Ss     0:00 ssh-agent
16118 ?        Ss     0:00 ssh-agent
16185 pts/5    S+     0:00 grep ssh-agent
27962 ?        S      0:00 /usr/bin/ssh-agent -D -a /run/user/710624/keyring/.ssh

So… why can’t LibGit2 see it?

6

I found a magic incantation online. I don’t know what this does, but it worked:

~/.ssh % eval $(ssh-agent) > /dev/null
~/.ssh % ssh-add id_ed25519
~/.ssh % ssh-add -l
256 SHA256:GBqs+SzszuFd8PdgDo34UYzOVGSgIIQ6QuI6ecyJ4bA duc299@e1-052367 (ED25519)
...
julia> using Pkg

julia> Pkg.clone("git@github.com:eford/CORBITS.git")
┌ Warning: Pkg.clone is only kept for legacy CI script reasons, please use `add`
└ @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:445
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `git@github.com:eford/CORBITS.git`
  Updating git-repo `git@github.com:eford/CORBITS.git`
[ Info: Assigning UUID 811cb5c5-b78b-5ed9-b25f-8f0cd4e719fe to CORBITS
 Resolving package versions...
  Updating `~/.julia/environments/v0.7/Project.toml`
  [811cb5c5] + CORBITS v0.0.0 [`~/.julia/dev/CORBITS`]
  Updating `~/.julia/environments/v0.7/Manifest.toml`
  [811cb5c5] + CORBITS v0.0.0 [`~/.julia/dev/CORBITS`]

So the immediate issue is resolved. It’d be good to not have to go through this, or at least understand what exactly I just did. Just running ssh-agent on the shell didn’t work. That “eval” bit seems to be required.

4 Likes
7

Yes,
that is how ssh-agent words.
It isn’t nice, but it is how it works for everyone.

(I am not a fan of ssh-agent)

3 Likes
8

Has anyone ever found any way to get this to use the keys properly? Is this a bug in LibGit2? It very much appears to be, as I have had this problem with multiple repos from multiple sources.

3 Likes
9

I got this problem too. I don’t quite understand the solution post …

1 Like
10

The stated solution in this thread never worked for me.

I have looked more into resolving this since my last posting. It still appears that there is no way of resolving it. From what I gather, this is simply a libgit2 bug, so there’s quite simply no way of getting the package manager to use private repos right now other than re-compiling Julia linked to a more recent version of libgit2. This is a major and ongoing issue for me, because I am reduced to using increasingly elaborate pull scripts rather than the package manager for private packages (and then I have to use dev RELATIVE_PATH to add the pulled repos). It’s a really bad situation.

I know that Julia 1.2 upgraded the libgit2 dependency, but the version it uses is still very old (the libgit2 repo moves very fast). I’m really hoping that the updated version will resolve this issue, somehow. If it doesn’t, I fear that I will wind up with a script so elaborate that it basically reproduces half the functionality of the package manager by the time the issue is fixed.

3 Likes
11

I’m not sure what is going wrong in your case
but I have regularly used julia 0.6, 0.7, 1.0, and 1.1 with private repos, using the prebuild binary.
via dev or add by URL (clone in 0.6).
with ssh-agent.
(other options beyond ssh-agent do not work. for people following at home see SSH auth keys, just very painful outside of ssh-agent. · Issue #911 · JuliaLang/Pkg.jl · GitHub )

12

I think it was you who told me that it only works if your key is in ~/.ssh/id_rsa and that only the command line ssh recognizes the config file. I still haven’t tested this. I wasn’t able to get it to work with ssh-agent the last time I tried it, but that was a while ago, so I hope to explore all of this again soon.

13

and ssh-agent.
Recently it seems like I wasn’t able to get id_rsa to work but @iamed2 has.

and that only the command line ssh recognizes the config file.

definately true.

14

FYI I created a PR Feature request: Allow path starting with ~/ to be used as a repo-url by tkf · Pull Request #1220 · JuliaLang/Pkg.jl · GitHub to let you use local path starting with ~/ as a repository URL. I think it works as a convenient ad-hoc solution for private repositories when SSH key does not work or just tedious to set it up.

15

This is frustrating. What’s weird is that for me pkg> dev was working with private repos via my private ssh key fine on one computer. But now I am trying to use this on a different computer and it’s not working (it just keeps asking for the private key location, even though its in ~/.ssh/id_rsa).

2 Likes
16

I’ve seen the same problem. When trying to dev from a private repo url, it works fine using my ssh keys on some computers, but others give me an error that it can’t find them even though the environment for my ssh keys is identical.

This problem is extremely frustrating and there’s some machines where its impossible to work because of this. Even when setting up environment variables, or using an ssh-agent to try to work around the problem, it still will not clone using dev/add in the package manager. I wish this problem would receive more attention and help to be solved.

17

So it’s not just me seeing this work on some computers but not others. I think it’s weird.
From what I understand Julia is using its own bundled LibGit2. Why would this issue depend on the system?

18

I’ve tried on systems where both are using the same version of precompiled julia (with bundled libgit2) and one works and the other doesn’t. So it may have to do with libgit2, but there must be something else in the environment causing it to only work sometimes.

19

You have to run this once? Or every time you login?

20

I just tried this, and it works pretty well for a Juno workflow
Set “Julia Options” > “Arguments” to:

SSH_PUB_KEY_PATH=~/.ssh/key.pub, SSH_KEY_PATH=~/.ssh/key

Outside of Juno, julia just needs to be started up with those variables:

$ SSH_PUB_KEY_PATH=~/.ssh/key.pub SSH_KEY_PATH=~/.ssh/key julia

or set them in ~/.bashrc

export SSH_PUB_KEY_PATH=~/.ssh/key.pub
export SSH_KEY_PATH=~/.ssh/key

It’s an extra step but not too bad.

Edit: This will also require switching Pkg to ssh mode, via

Pkg.setprotocol!(protocol=“ssh”)
2 Likes