Github CI with private repository in Julia

Dear Community,

I am having problems on a CI of a private repository of mine, the following error arises during the buildkpg phase of the CI:

caused by: GitError(Code:EUSER, Class:Callback, Aborting, user cancelled credential request.)

The reason is due to the fact that in the building phase of the environment, I do have the https link to the private package (https://github.com//.jl), but it seems like in the process, the CI does not recognize that it is run from my package.

How can I solve?

Thank you

The CI is the standard one:

name: CI
on:
  push:
    branches: [main]
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  test:
    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - version: '1'
            os: ubuntu-latest
            arch: x64
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@v1
        with:
          version: ${{ matrix.version }}
          arch: ${{ matrix.arch }}
      - uses: actions/cache@v1
        env:
          cache-name: cache-artifacts
        with:
          path: ~/.julia/artifacts
          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
          restore-keys: |
            ${{ runner.os }}-test-${{ env.cache-name }}-
            ${{ runner.os }}-test-
            ${{ runner.os }}-
      - uses: julia-actions/julia-buildpkg@v1
      - uses: julia-actions/julia-runtest@v1
      #- uses: julia-actions/julia-processcoverage@v1
      #- uses: codecov/codecov-action@v1
      #  with:
      #    file: lcov.info

What code are you trying to run exactly in that “building phase of the environment”? Can you share?

The toml is the following:

name = ...
uuid = ...
authors = ...
version = "0.1.0"

[deps]
MYPRIVATEREPO = ...
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
...

[compat]
MathOptInterface = "1.0"
julia = "1"

and in the manifest, except the standard values, there is:

[[MYPRIVATEREPO ]]
deps = ...
git-tree-sha1 = ...
repo-rev = "main"
repo-url = "https://github.com/<github_id>/<MYPRIVATEREPO>.jl"
uuid = ...
version = "0.1.0"

So, you have a package, say, Foo.jl and you also have Foo.jl in the [deps] section of the Project.toml? Or do you have another package that refers to your private package Foo.jl?

Generally, what you could do in situations where there is a private repository inside the Manifest.toml is to install it before running buildpkg. For example:

  - run: julia --project -e 'using Pkg; Pkg.add(; url="https://...")'
  - uses: julia-actions/julia-buildpkg@v1

getting access to the repository inside the GitHub Action is possible. You can use a GitHub Action like webfactory/ssh-agent@v0.5.4 to set a secrets.DEPLOY_KEY from your private repository to give GitHub Actions access from one private repository to another.

EDIT: To avoid confusion. What I mean by DEPLOY_KEY is that you add a key to the deploy keys of your private repository and put this key as a secret in the repository in which you currently have problems. Then, set that secret via webfactory/ssh-agent before running the Pkg.add.

I’ve tried, but I’m still having issues:

Run webfactory/ssh-agent@v0.5.4
Adding GitHub.com keys to /home/runner/.ssh/known_hosts
Starting ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-lN2HPXrnFuVC/agent.1665
SSH_AGENT_PID=1666
Adding private key(s) to agent
Identity added: (stdin) (git@github.com:davide-f/<repo>.jl.git)
Key(s) added:
4096 SHA256:yAtLh9RzQGzTIGFpDat7iegtJKxD1zBF1I+33laqnE8 git@github.com:davide-f/<repo>.jl.git (RSA)
Configuring deployment key(s)
Added deploy-key mapping: Use identity '/home/runner/.ssh/key-...' for GitHub repository davide-f/<repo>.jl
Comment for (public) key '' does not match GitHub URL pattern. Not treating it as a GitHub deploy key.

The new CI.yaml is:

name: CI
on:
  push:
    branches: [main]
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  test:
    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - version: '1'
            os: ubuntu-latest
            arch: x64
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@v1
        with:
          version: ${{ matrix.version }}
          arch: ${{ matrix.arch }}
      - uses: webfactory/ssh-agent@v0.5.4
        with:
            ssh-private-key: ${{ secrets.SSH_SECRET }}
      - uses: actions/cache@v1
        env:
          cache-name: cache-artifacts
        with:
          path: ~/.julia/artifacts
          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
          restore-keys: |
            ${{ runner.os }}-test-${{ env.cache-name }}-
            ${{ runner.os }}-test-
            ${{ runner.os }}-
...

I’ve followed the guide ssh-agent/README.md at master · webfactory/ssh-agent · GitHub

I put the public key into the deploy keys (named github.com) of “other” private github repo that has to be installed from another private repo

ssh-rsa  <...> git@github.com:davide-f/<repo>.jl.git

The private key has been added into the secret named “SSH_SECRET” of the repo2 that should install the repo1

Still, however, I have problems

Any idea?

Thank you for your help

Yes: I have two private repos, and I am trying from repo2 to install repo1

Can you be more specific? Still access denied errors?

Yes, see the message before, basically the error is the last line of the first block:

Comment for (public) key '' does not match GitHub URL pattern. Not treating it as a GitHub deploy key.

It seems like the keys are not recognized somehow, though it seems to me that they should be fine.

This error suggests that the SSH is not successful and as a consequence, the error I was experiencing during the building of the environment is not solved.
The subsequent error is as follows:

ERROR: failed to clone from https://github.com/davide-f/<repo>.jl, error: GitError(Code:ERROR, Class:Net, failed to resolve address for key-cd9edd77eda3f8efb9b9586183a4d3d26afe042eb1e7e8b23e8b41238d3ef161.github.com: Name or service not known)

Ah. Now I see the error. Have you looked at Multiple deploy keys not working · Issue #88 · webfactory/ssh-agent · GitHub? I’ve setup multiple repositories with SSH keys deploying from one to the other. Most recently a few days ago, so it should work. Double check that you set the right keys.

Thank you, I solved.
In particular, I cloned the repo first and then installed it separately.
Thank you very much!

1 Like

I’m having the same issue and I can’t figure out how to get the webfactory ssh-agent to work for multiple keys to different private repos. Would you mind sharing what your CI.yml looks like now with the fix you mentioned above?

Here it is

name: CI
on:
  push:
    branches: [main]
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  test:
    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - version: '1'
            os: ubuntu-latest
            arch: x64
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@v1
        with:
          version: ${{ matrix.version }}
          arch: ${{ matrix.arch }}
      - uses: webfactory/ssh-agent@v0.5.4
        with:
            ssh-private-key: ${{ secrets.SSH_SECRET }}
      - name: Clone Games.jl
        run: git clone https://github.com/davide-f/{MyPrivateRepo}.jl
      - name: Install private repo
        run: julia --project -e 'using Pkg; Pkg.add(; url="{MyPrivateRepo}.jl")'
      - uses: actions/cache@v1
        env:
          cache-name: cache-artifacts
        with:
          path: ~/.julia/artifacts
          key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
          restore-keys: |
            ${{ runner.os }}-test-${{ env.cache-name }}-
            ${{ runner.os }}-test-
            ${{ runner.os }}-
      - uses: julia-actions/julia-buildpkg@v1
      - uses: julia-actions/julia-runtest@v1
1 Like

Thank you!