How to add package from a GitHub repo's subfolder

General Usage
,
1

A same question has been answered at Installing from a subdirectory of Github.
However, I still cannot make it work. In my repo GitHub - ShuhuaGao/GADNF: Reconstruct Boolean network from data using GA on its DNF, there is a subfolder “src/GADNF.jl” which forms a package. I try to add it with the following:

Pkg.add(url="https://github.com/ShuhuaGao/GADNF", subdir="src/GADNF.jl")

The error is

Updating git-repo `https://github.com/ShuhuaGao/GADNF`
ERROR: expected the file `src/GADNF.jl` to exist for package `GADNF` at `C:\Users\shuhu\AppData\Local\Temp\jl_sscy2b`
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:70
  [2] read_package(path::String)
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:580
  [3] resolve_projectfile!(env::Pkg.Types.EnvCache, pkg::Pkg.Types.PackageSpec, project_path::String)
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:862
  [4] (::Pkg.Types.var"#57#58"{Pkg.Types.Context, Pkg.Types.PackageSpec, String})(repo::LibGit2.GitRepo)
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:832
  [5] with(f::Pkg.Types.var"#57#58"{Pkg.Types.Context, Pkg.Types.PackageSpec, String}, obj::LibGit2.GitRepo)
    @ LibGit2 C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\LibGit2\src\types.jl:1160
  [6] handle_repo_add!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:782
  [7] handle_repos_add!(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.Types C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:852
  [8] add(ctx::Pkg.Types.Context, pkgs::Vector{…}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::@Kwargs{…})
    @ Pkg.API C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:258
  [9] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::@Kwargs{})
    @ Pkg.API C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:159
 [10] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:148
 [11] add
    @ C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:146 [inlined]
 [12] add(; name::Nothing, uuid::Nothing, version::Nothing, url::String, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::String, kwargs::@Kwargs{})
    @ Pkg.API C:\Users\shuhu\.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:176
 [13] top-level scope
    @ REPL[24]:1
Some type information was truncated. Use `show(err)` to see complete types.

Any suggestion? Besides, how shall I do the same operation in REPL pkg mode, e,g, add ....

2

The Pkg REPL invocation is

pkg> add https://github.com/ShuhuaGao/GADNF:src/GADNF.jl

This (and your Pkg.add call) works on the first try but reproduces your failure if it is repeated. If I remove the corresponding entry in ~/.julia/clones it again works, but still only once.

So it looks like a Pkg bug. I recommend opening an issue at GitHub - JuliaLang/Pkg.jl: Pkg - Package manager for the Julia programming language.

3

Thanks. I cleaned the ~/.julia/clones, and ran pkg> add https://github.com/ShuhuaGao/GADNF:src/GADNF.jl. But it was stuck on “cloning” forever.

(jl_Ld4ZeV) pkg>  add https://github.com/ShuhuaGao/GADNF:src/GADNF.jl
     Cloning git-repo `https://github.com/ShuhuaGao/GADNF`

julia> versioninfo()
Julia Version 1.10.4
Commit 48d4fd4843 (2024-06-04 10:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 16 × 13th Gen Intel(R) Core(TM) i5-13500H
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, goldmont)
Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
Environment:
  JULIA_NUM_THREADS = auto
4

Cloning took less than a second for me, so that’s either a networking problem or a different behavior between Windows and Linux. Either way it’s a separate issue from the original one.

Things to check for the cloning problem:

  • Can you add any other package by URL?
  • Can you git clone your repo outside of Julia?
  • Does it make any difference if you set the environment variable JULIA_PKG_USE_CLI_GIT to true?
1 Like
5
  • Can add Example.jl normally as GitHub - JuliaLang/Example.jl: Example Julia package repo.
  • Can git clone the GADNF repo
  • Now the error becomes
    (@v1.10) pkg> add https://github.com/ShuhuaGao/GADNF:src/GADNF.jl
ERROR: invalid git HEAD (reference 'refs/heads/master' not found)
    after I set JULIA_PKG_USE_CLI_GIT. The issue stays even if I remove the environment variable.
6

It seems your command assumes the default branch is master; however a few years back this default changed (for new repositories) and is not main, for example for that repository.

So according to here (did not test it) Add package with specific branch failing since julia 1.5.2 · Issue #2230 · JuliaLang/Pkg.jl · GitHub
its probably

(@v1.10) pkg> add https://github.com/ShuhuaGao/GADNF#main:src/GADNF.jl
7

Thanks for the hints. Unfortunately, neither worked below:

(jl_vxKllk) pkg>  add https://github.com/ShuhuaGao/GADNF#main:src/GADNF.jl
ERROR: invalid git HEAD (reference 'refs/heads/master' not found)

(jl_vxKllk) pkg>  add https://github.com/ShuhuaGao/GADNF#master:src/GADNF.jl
ERROR: invalid git HEAD (reference 'refs/heads/master' not found)

Nevertheless, it seems true that

your command assumes the default branch is master

Based on this observation, it turns out that the following statement indeed worked:

Pkg.add(url="https://github.com/ShuhuaGao/GADNF#main", subdir="src/GADNF.jl")

though I don’t know how to write the same as a Pkg command in REPL.

8

Well, I was hoping my adaption would work but at least the long version then did work,I did not find much of how to otherwise specify this for ] add, so I do not now that either.

The error message of your first command seems to indicate the branch is not recognised there (the #main part).