# CI for private repo repending on private repo on Gitlab

**URL:** <https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352>\
**Category:** Tooling\
**Tags:** question\
**Created:** [September 27, 2020, 7:57am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352 "2020-09-27T07:57:04Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 27, 2020, 7:57am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/1 "2020-09-27T07:57:05Z")

</div>

I have two private repos on Gitlab, lets call them Foo and Bar, the latter depends on Foo.

When developing on my local machine, I have Gitlab authentication set up, so I can just did

```julia
pkg> activate /path/to/Bar
pkg> add https://gitlab.com/tkpapp/Foo.jl

```

and everything works fine.

But I also need a solution for Gitlab CI, and so far I have not been able to figure one out. Gitlab offers [access tokens](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html), but they become part of the URL, so I would need a different manifest on CI. Submodules kind of work using an internal path on Gitlab (it’s the same user), but I need to overwrite my `.gitmodules` files for CI because it’s different on my local machine.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 27, 2020, 8:35am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/2 "2020-09-27T08:35:41Z")

</div>

Why not add the private packages via git (as in `pkg> add git@gitlab.com:USER/PACKAGE.jl.git`) so that you can use SSH key to control access?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 27, 2020, 8:50am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/3 "2020-09-27T08:50:44Z")

</div>

I could not get that to work (for reasons I gave up on investigating, something to do with Gitlab’s security model), but I found that Git subtrees work fine.

> **[Git Subtree: Alternative to Git Submodule | Atlassian Git Tutorial](https://www.atlassian.com/git/tutorials/git-subtree)**
>
> Git Subtree is an extension to Git that allows you to split up large projects into smaller ones. It's like a virtual repository but more powerful.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 27, 2020, 8:58am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/4 "2020-09-27T08:58:28Z")

</div>

Hmm… it worked for me between multiple private [GitLab.com](http://GitLab.com) repositories. I just followed [https://docs.gitlab.com/ee/ci/ssh\_keys/README.html](https://docs.gitlab.com/ee/ci/ssh_keys/README.html)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 27, 2020, 10:08am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/5 "2020-09-27T10:08:07Z")

</div>

Can you please share the relevant `.gitlab-ci.yml` file is possible?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 27, 2020, 8:37pm UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/6 "2020-09-27T20:37:40Z")

</div>

To refresh my memory, I set up a demo that shows the configuration [https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/merge\_requests/1](https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/merge_requests/1)

> Just following [https://docs.gitlab.com/ee/ci/ssh\_keys/README.html](https://docs.gitlab.com/ee/ci/ssh_keys/README.html)
> 
> Concrete steps:
> 
> 1. Create an SSH key pair with `ssh-keygen -t ed25519 -C GitLabDemoDownstream@gitlab-ci -f key`
> 2. Add the public key as a deploy key at [https://gitlab.com/tkfm/gitlabdemoupstream.jl/-/settings/repository#js-deploy-keys-settings](https://gitlab.com/tkfm/gitlabdemoupstream.jl/-/settings/repository#js-deploy-keys-settings)
> 3. Add the private key as `SSH_PRIVATE_KEY` environment variable at [https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/settings/ci\_cd](https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/settings/ci_cd)
> - Turn _off_ `Protected` switch.
> - Without this, `$SSH_PRIVATE_KEY` is not set and `ssh-add` would complain that `Error loading key "(stdin)": invalid format` [Julia 1.5 (#759713949) · Jobs · Takafumi Arakaki / GitLabDemoDownstream.jl · GitLab](https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/jobs/759713949#L107)
> 
> — [https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/merge\_requests/1](https://gitlab.com/tkfm/gitlabdemodownstream.jl/-/merge_requests/1)

The CI for the merge request was run before making these repositories public.

I also invited you to the repositories so that you can see the settings.

* * *

Edit: `Protected` switch setting would look like:

> ![image](https://global.discourse-cdn.com/julialang/original/3X/6/3/63b0acf56fea5334faa466588454974210ca67a3.png)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 28, 2020, 7:11am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/7 "2020-09-28T07:11:30Z")

</div>

Thank you so much! I think the issue for me was the _Protected_ switch.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 28, 2020, 7:25am UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/8 "2020-09-28T07:25:32Z")

</div>

You are welcome! Yeah, I think it might be a common pitfall. I totally forgot about it.

---

<div class="post-metadata">

**Author:** ![judober](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/judober/32/6676_2.png) [@judober](https://discourse.julialang.org/u/judober)\
**Post date:** [April 7, 2021, 10:49pm UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/9 "2021-04-07T22:49:12Z")

</div>

I used the guide posted above in order to setup my CI which worked fine (thanks by the way!) until recently. After Updating to Julia 1.6 I get:

```julia
SSH host verification: the server `private.gitlab:22` is not a known host. Please run `ssh-keyscan private.gitlab:22 >> /root/.ssh/known_hosts` in order to add the server to your known hosts file and then try again.

```

However, manually adding this server in the .yml does not work.

I found [https://github.com/JuliaLang/julia/issues/38777](https://github.com/JuliaLang/julia/issues/38777) which seems related. At one point it is suggested that adding `ssh://` in front of `git.private.gitlab` might fix the problem. However, when I tried this in the `Pkg.registry.add`-command, CI could not resolve the url.

As you are using similar setups, did you run into this problem too? Could you solve it?

Edit: I think I found the problem [https://github.com/JuliaLang/julia/issues/40297](https://github.com/JuliaLang/julia/issues/40297)

Adding this solved my problem: `ssh-keyscan private.gitlab >> /root/.ssh/known_hosts`

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 9, 2021, 12:20pm UTC](https://discourse.julialang.org/t/ci-for-private-repo-repending-on-private-repo-on-gitlab/47352/10 "2021-04-09T12:20:03Z")

</div>

I think it is best to wait for the upstream issues to resolve, in the meantime use the workaround.