Can no longer use private repository on Github as dependency

Tooling Package Management
1

Hello everyone. I’m quite new to Julia and I’m currently developing a package that uses several dependencies, among which are CUDA and a core package I developed myself and put on Github (basically the package is a GPU fork of the core one).

I started initial development by creating a new project with generate and adding the dependencies to the environment. I added the repository on github by just doing add <link to github repository>, and that worked fine.
Today while testing I noticed CUDA had a new update available, so I decided to run an up command, however when it parsed my repository it gave a cryptic error

pkg> up
    Updating registry at `~/.julia/registries/General.toml`
    Updating git-repo `<github repository url>`
Username for 'https://github.com': <my github username>
                                            ERROR: failed to fetch from <github repository url>, error: GitError(Code:ERROR, Class:HTTP, unexpected http status code: 403)

And nothing else. I tried to look online but found nothing useful, so I donwloaded gh and did the login to Github from the command line. That worked, and I regenerated the access token too. I attempted a second time and the same error occurred.
At this point I realized I had pushed a few new commits to my core package, so maybe something broke along the way. I then removed the dependency from the GPU package and attempted to add it again, but now I got a new error (though it’s also very cryptic)

add <github repository url>
    Updating git-repo <github repository url>
ERROR: failed process: Process(`/usr/bin/gh auth git-credential store`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:598 [inlined]
  [2] open(f::LibGit2.var"#94#96"{LibGit2.GitCredential}, cmds::Cmd, args::String; kwargs::@Kwargs{})
    @ Base ./process.jl:470
  [3] open
    @ ./process.jl:428 [inlined]
  [4] run!
    @ ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/gitcredential.jl:186 [inlined]
  [5] run
    @ ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/gitcredential.jl:201 [inlined]
  [6] approve
    @ ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/gitcredential.jl:210 [inlined]
  [7] approve(cfg::LibGit2.GitConfig, cred::LibGit2.UserPasswordCredential, url::String)
    @ LibGit2 ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/gitcredential.jl:290
  [8] approve(p::LibGit2.CredentialPayload; shred::Bool)
    @ LibGit2 ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/types.jl:1447
  [9] approve
    @ ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/types.jl:1435 [inlined]
 [10] fetch(repo::LibGit2.GitRepo; remote::String, remoteurl::SubString{String}, refspecs::Vector{String}, credentials::Nothing, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})
    @ LibGit2 ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/LibGit2.jl:306
 [11] fetch
    @ ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/LibGit2.jl:272 [inlined]
 [12] fetch(io::IOContext{IO}, repo::LibGit2.GitRepo, remoteurl::String; header::Nothing, credentials::Nothing, refspecs::Vector{String}, kwargs::@Kwargs{})
    @ Pkg.GitTools ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/GitTools.jl:178
 [13] (::Pkg.Types.var"#58#59"{Pkg.Types.Context, Pkg.Types.PackageSpec, String})(repo::LibGit2.GitRepo)
    @ Pkg.Types ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:815
 [14] with(f::Pkg.Types.var"#58#59"{Pkg.Types.Context, Pkg.Types.PackageSpec, String}, obj::LibGit2.GitRepo)
    @ LibGit2 ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/LibGit2/src/types.jl:1166
 [15] handle_repo_add!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)
    @ Pkg.Types ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:790
 [16] handle_repos_add!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.Types ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:860
 [17] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, target::Symbol, allow_autoprecomp::Bool, kwargs::@Kwargs{io::IOContext{IO}})
    @ Pkg.API ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:285
 [18] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{})
    @ Pkg.API ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
 [19] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:148
 [20] do_cmd(command::Pkg.REPLMode.Command, io::Base.TTY)
    @ Pkg.REPLMode ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/REPLMode/REPLMode.jl:407
 [21] do_cmds(commands::Vector{Pkg.REPLMode.Command}, io::Base.TTY)
    @ Pkg.REPLMode ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/REPLMode/REPLMode.jl:393
 [22] do_cmds(repl::REPL.LineEditREPL, commands::String)
    @ REPLExt ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/ext/REPLExt/REPLExt.jl:92
 [23] on_done(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool, repl::REPL.LineEditREPL)
    @ REPLExt ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/ext/REPLExt/REPLExt.jl:106
 [24] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [25] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [26] (::REPLExt.var"#47#50"{REPL.LineEditREPL})(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool)
    @ REPLExt ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/ext/REPLExt/REPLExt.jl:128
 [27] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [28] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [29] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/LineEdit.jl:2755
 [30] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1474
 [31] (::REPL.var"#75#81"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:480

To me this looks like some issue with the credentials, but I can clearly push and pull and gh auth status gives no error. I even instantiated my token with all possible scopes, so I really don’t understand what’s going on.

If you have any idea, it would be greatly appreciated.

2

There used to be an environment variable JULIA_PKG_USE_SYSTEM_GIT, or similar, can‘t find it now in the docs, that I had to set in order to be able to use private repos.

3

Ah here it is JULIA_PKG_USE_CLI_GIT

3 Likes
4

It is also possible to use SSH authentication instead.

  • Setup SSH authentication on github and you computer
  • Maybe you still have to change JULIA_PKG_USE_CLI_GIT
  • Clone the package with ] add git@github.com:user/privatePackage.jl

It is a bit annoying to setup but afterwards the private package works just like an unregistered public package and you don’t have to enter your credentials all the time.

5

Thank you very much! Setting that to true worked like a charm. Next time I’ll be more mindful to check environment variables when everything else fails

1 Like
6

Setting JULIA_PKG_USE_CLI_GIT to true has the added benefit that the CLI git client also respects your git_config, so setting usernames and ports, for example, will have their desired effects. The internally linked libgit2 solution doesn’t do that.

1 Like