Codecov.io and using packages not in the general registry

I’ve created a package that uses another package which is not in the general registry. When the CI/coverage testing runs (github action) it fails, saying it expects the included package to be found in the general registry. (Error message below.)

I imagine there is some way (in the yaml file) to tell the CI runner to add a package by URL instead for looking in the general registry. I tried something like this in my CI.yml file:

- run: julia -e 'import Pkg; Pkg.add(url="https://github.com/wildart/SmithNormalForm.jl")' but I still get the same error. Somehow the added package isn’t persisting from one line in the yaml file to the next. This is probably a simple problem, but I’ve tried multiple variations with no success. Anyone have some advice?

Here is my full yaml file:

name: Runtests
on: [push, pull_request]
jobs: 
  test: 
    runs-on: ${{ matrix.os }}
    strategy:
      matrix: 
        julia-version: ['1.8']
        julia-arch: [x64]
        os: [ubuntu-latest, macOS-latest, windows-latest]
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@latest
        with: 
          version: ${{ matrix.julia-version }}
      - run: julia -e 'import Pkg; Pkg.add(url="https://github.com/wildart/SmithNormalForm.jl")'
      - uses: julia-actions/julia-buildpkg@latest
      - uses: julia-actions/julia-runtest@latest
      - uses: julia-actions/julia-processcoverage@v1
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}      
ERROR: LoadError: expected package `SmithNormalForm [ba71f38f]` to be registered
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] check_registered
   @ ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:1190 [inlined]
 [3] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_build::Bool, allow_autoprecomp::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1562
 [4] build(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}, verbose::Bool)
   @ Pkg.Operations ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:895
 [5] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; verbose::Bool, kwargs::Base.Pairs{Symbol, IOContext{Base.PipeEndpoint}, Tuple{Symbol}, NamedTuple{(:io,), Tuple{IOContext{Base.PipeEndpoint}}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1029
 [6] build(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{Base.PipeEndpoint}, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:156
 [7] build(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:171
 [8] (::Base.var"#88#90"{Base.var"#88#89#91"{ExponentialBackOff, Nothing, typeof(Pkg.API.build)}})(; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Base ./error.jl:309
 [9] top-level scope
   @ ~/work/_temp/906d5260-4813-4de2-b3c2-ac1428229c10:27
in expression starting at /Users/runner/work/_temp/906d5260-4813-4de2-b3c2-ac1428229c10:27

caused by: expected package `SmithNormalForm [ba71f38f]` to be registered
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] check_registered
   @ ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:1190 [inlined]
 [3] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_build::Bool, allow_autoprecomp::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1556
 [4] build(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}, verbose::Bool)
   @ Pkg.Operations ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:895
 [5] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; verbose::Bool, kwargs::Base.Pairs{Symbol, IOContext{Base.PipeEndpoint}, Tuple{Symbol}, NamedTuple{(:io,), Tuple{IOContext{Base.PipeEndpoint}}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1029
 [6] build(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{Base.PipeEndpoint}, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:156
 [7] build(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Pkg.API ~/hostedtoolcache/julia/1.8.5/aarch64/share/julia/stdlib/v1.8/Pkg/src/API.jl:171
 [8] (::Base.var"#88#90"{Base.var"#88#89#91"{ExponentialBackOff, Nothing, typeof(Pkg.API.build)}})(; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}})
   @ Base ./error.jl:309
 [9] top-level scope
   @ ~/work/_temp/906d5260-4813-4de2-b3c2-ac1428229c10:27
Error: Process completed with exit code 1.

I was able to get this to work. I relied on chatGPT4o heavily for suggestions. I’m not sure completely why this works, but it seems to be necessary to remove one of the unregistered packages and then reinstall it each time the github action installs my package and dependencies. The tests take a long time now (lots of precompilation). But at least they actually succeed in running.

I would love feedback/explanation of why its necessary to rm and reinstall one of my unregistered dependencies…

Here is my working CI.yaml file:

name: Runtests
on: [push, pull_request]
# needed to allow julia-actions/cache to delete old caches that it has created
permissions: 
  actions: write
  contents: read
jobs: 
  test: 
    runs-on: ${{ matrix.os }}
    strategy:
      matrix: 
        julia-version: ['1.9']
        julia-arch: [x64]
        os: [ubuntu-latest, macOS-latest, windows-latest]
        #os: [macOS-latest]
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@latest
        with: 
          version: ${{ matrix.julia-version }}
      #- uses: julia-actions/cache@v2
      - run: |
          julia -e '
            import Pkg;
            Pkg.activate(".");  # Activate the project environment
            # Not sure why removing and reinstalling the SNF package is necessary, but it is
            Pkg.rm("SmithNormalForm");  # Remove any existing installation of SNFer
            Pkg.add(url="https://github.com/glwhart/Spacey.jl"); # Re-add Spacey from the URL
            Pkg.add(url="https://github.com/wildart/SmithNormalForm.jl");  # Add the unregistered SNF package
          '
      - uses: julia-actions/julia-buildpkg@latest
      - uses: julia-actions/julia-runtest@latest
      - uses: julia-actions/julia-processcoverage@v1
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}