# Julia and Gitlab self-hosted : a state-of-the-art?

**URL:** <https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685>\
**Category:** General Usage\
**Tags:** packages, gitlab, registry\
**Created:** [September 2, 2022, 11:54am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685 "2022-09-02T11:54:59Z")\
**Posts on this page:** 12\
**Page:** 2

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [April 18, 2023, 4:39pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/21 "2023-04-18T16:39:48Z")

</div>

@ArnaudHenry Did you try by any chance, to download an artifact using a token from your private Gitlab ?  
I’ve been trying to configure package artifacts, but it seems the only way to connect is to use a token in the header of the request.

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [May 14, 2023, 10:10am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/22 "2023-05-14T10:10:44Z")

</div>

I would recommend to use [Releases | GitLab](https://docs.gitlab.com/ee/user/project/releases/) (with a [generic package](https://docs.gitlab.com/ee/user/packages/generic_packages/index.html)) for artifacts, i.e., with a separate GitLab CI job in the GitLab CI pipeline to circumvent the limitation of Artifacts not supporting authentication - by downloading the artifacts to a local (cached) path before the Julia jobs needing the artifacts run (and re-writing the paths of the artifacts to use the local ones) - e.g. using [GitForge.jl](https://github.com/JuliaWeb/GitForge.jl) if possible - or [python-gitlab](https://python-gitlab.readthedocs.io/en/stable/).

I have yet to implement this, though - but I believe it should work 😇 🙂

I would support implementing this in a common Julia GitLab CI Templates repo., e.g. [IHP Systems / Julia / Julia GitLab CI templates · GitLab](https://gitlab.com/IHPSystems/julia/julia-gitlab-ci-templates) (which could be moved to a common JuliaCI org.) - the aim of which is to support generic Julia package development on GitLab CI, including:

- Support for different project types, e.g., packages, applications, registries, BinaryBuilder recipes, packages with Python dependencies, (_and packages using private artifacts_).
- Support for best practices, e.g. testing (with multi-threading), formatting with JuliaFormatter, and analysis with JET, and Aqua.
- Support for GitLab CI features like test reports, caching, “needs”, etc.
- _Extended_ support for Julia versions (e.g. the last two LTS versions, and the last two stable versions) - to support CI even for projects migrating from older to newer versions, e.g. from Julia 1.0 to Julia 1.6.
- Support for Julia-supported platfoms (across architectures and operating systems) - using tags compatible with [GitLab.com](http://GitLab.com) SaaS runners (a super set of those runner tags).
- Versioned and stable - respecting semantic versioning.

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [May 15, 2023, 11:43am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/23 "2023-05-15T11:43:13Z")

</div>

Thanks for your proposal, looking forward to seeing that in action !  
In the meantime, I finally found a way to configure artifacts that works for my needs. See [How to configure an artifact to use a header during package download? - #2 by BambOoxX](https://discourse.julialang.org/t/how-to-configure-an-artifact-to-use-a-header-during-package-download/96057/2)

---

<div class="post-metadata">

**Author:** ![stemann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stemann/32/4030_2.png) [@stemann](https://discourse.julialang.org/u/stemann)\
**Post date:** [May 15, 2023, 11:49am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/24 "2023-05-15T11:49:50Z")

</div>

Much of it is already there - but of course not the packages with private artifacts yet 🙂 Will update when I get to it.

If you are working with BinaryBuilder stuff I have also hacked some stuff together with `sed` to build, deploy locally and test in `build_binaries` and `test_binaries` in [pylon\_julia\_wrapper/.gitlab-ci.yml at master · IHPSystems/pylon\_julia\_wrapper · GitHub](https://github.com/IHPSystems/pylon_julia_wrapper/blob/master/.gitlab-ci.yml#L19) (and mirrored in the corresponding GitHub Actions workflow).

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [July 10, 2023, 11:53am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/25 "2023-07-10T11:53:17Z")

</div>

@ArnaudHenry @oxinabox , something has been bothering me recently. It seems I can’t display coverage for each file of my repo in a merge request diff. I’m using [`LocalCoverage.jl`](https://github.com/JuliaCI/LocalCoverage.jl) to generate the coverage report as a Cobertura XML file. But I can’t figure if this file is properly used by the Gitlab.  
Did you encounter such problems at Invenia ?

---

<div class="post-metadata">

**Author:** ![disberd](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@disberd](https://discourse.julialang.org/u/disberd)\
**Post date:** [July 10, 2023, 3:42pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/26 "2023-07-10T15:42:59Z")

</div>

@BambOoxX I also have some packages on a self-hosted gitlab instance (not Invenia).  
I do see the coverage of files that were changed within a merge request by uploading the XML generated using LocalCoverags.jl, but I can’t (or don’t know how) see the coverage of files that were not touched by the MR commits.

Is this the problem you are refering to, or you don’t see coverage at all?

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [July 11, 2023, 7:37am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/27 "2023-07-11T07:37:59Z")

</div>

@disberd Hmm I do not see both ^^. But first priority could be to see the coverage in the MR.  
Could you share a samble of a Cobertura file as well as you .gitlab-ci file for coverage ?

---

<div class="post-metadata">

**Author:** ![disberd](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@disberd](https://discourse.julialang.org/u/disberd)\
**Post date:** [July 11, 2023, 10:27am UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/28 "2023-07-11T10:27:51Z")

</div>

@BambOoxX,

Here is a the cobertura sample

> **cobertura-coverage.xml**
>
> ```xml
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
> <coverage branch-rate="0.0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.5056179775280899" lines-covered="45" lines-valid="89" timestamp="1689070463" version="2.0.3">
> <sources>
> <source>.</source>
> </sources>
> <packages>
> <package line-rate="0.5056179775280899" branch-rate="0.0" name="src" complexity="0">
> <classes>
> <class branch-rate="0.0" complexity="0" filename="src/ActiveAntenna.jl" line-rate="1.0" name="src.ActiveAntenna.jl">
> <methods/>
> <lines>
> <line branch="false" hits="6" number="1"/>
> <line branch="false" hits="1" number="34"/>
> <line branch="false" hits="1" number="36"/>
> </lines>
> </class>
> <class branch-rate="0.0" complexity="0" filename="src/basic_types.jl" line-rate="0.5211267605633803" name="src.basic_types.jl">
> <methods/>
> <lines>
> <line branch="false" hits="44" number="41"/>
> <line branch="false" hits="41857" number="50"/>
> <line branch="false" hits="0" number="52"/>
> <line branch="false" hits="0" number="53"/>
> <line branch="false" hits="0" number="54"/>
> <line branch="false" hits="0" number="55"/>
> <line branch="false" hits="0" number="56"/>
> <line branch="false" hits="0" number="57"/>
> <line branch="false" hits="0" number="58"/>
> <line branch="false" hits="0" number="59"/>
> <line branch="false" hits="0" number="61"/>
> <line branch="false" hits="0" number="63"/>
> <line branch="false" hits="0" number="64"/>
> <line branch="false" hits="0" number="65"/>
> <line branch="false" hits="9" number="73"/>
> <line branch="false" hits="7688" number="74"/>
> <line branch="false" hits="17882" number="93"/>
> <line branch="false" hits="17882" number="94"/>
> <line branch="false" hits="17882" number="95"/>
> <line branch="false" hits="16295" number="117"/>
> <line branch="false" hits="16295" number="118"/>
> <line branch="false" hits="16295" number="119"/>
> <line branch="false" hits="36" number="143"/>
> <line branch="false" hits="36" number="144"/>
> <line branch="false" hits="36" number="145"/>
> <line branch="false" hits="20" number="152"/>
> <line branch="false" hits="12" number="153"/>
> <line branch="false" hits="8" number="154"/>
> <line branch="false" hits="0" number="155"/>
> <line branch="false" hits="26" number="157"/>
> <line branch="false" hits="18" number="159"/>
> <line branch="false" hits="35" number="309"/>
> <line branch="false" hits="1" number="310"/>
> <line branch="false" hits="26" number="311"/>
> <line branch="false" hits="1" number="313"/>
> <line branch="false" hits="187" number="316"/>
> <line branch="false" hits="0" number="317"/>
> <line branch="false" hits="376" number="318"/>
> <line branch="false" hits="0" number="324"/>
> <line branch="false" hits="0" number="325"/>
> <line branch="false" hits="0" number="326"/>
> <line branch="false" hits="0" number="327"/>
> <line branch="false" hits="102" number="330"/>
> <line branch="false" hits="102" number="340"/>
> <line branch="false" hits="0" number="341"/>
> <line branch="false" hits="0" number="353"/>
> <line branch="false" hits="0" number="354"/>
> <line branch="false" hits="0" number="355"/>
> <line branch="false" hits="0" number="356"/>
> <line branch="false" hits="0" number="357"/>
> <line branch="false" hits="0" number="358"/>
> <line branch="false" hits="0" number="359"/>
> <line branch="false" hits="0" number="360"/>
> <line branch="false" hits="0" number="361"/>
> <line branch="false" hits="0" number="362"/>
> <line branch="false" hits="20" number="392"/>
> <line branch="false" hits="8" number="418"/>
> <line branch="false" hits="10" number="446"/>
> <line branch="false" hits="36" number="458"/>
> <line branch="false" hits="66" number="459"/>
> <line branch="false" hits="36" number="460"/>
> <line branch="false" hits="36" number="461"/>
> <line branch="false" hits="36" number="464"/>
> <line branch="false" hits="36" number="465"/>
> <line branch="false" hits="36" number="466"/>
> <line branch="false" hits="0" number="468"/>
> <line branch="false" hits="0" number="469"/>
> <line branch="false" hits="0" number="470"/>
> <line branch="false" hits="0" number="473"/>
> <line branch="false" hits="0" number="474"/>
> <line branch="false" hits="4" number="497"/>
> </lines>
> </class>
> <class branch-rate="0.0" complexity="0" filename="src/lattices.jl" line-rate="0.3333333333333333" name="src.lattices.jl">
> <methods/>
> <lines>
> <line branch="false" hits="102" number="6"/>
> <line branch="false" hits="41872" number="8"/>
> <line branch="false" hits="41872" number="10"/>
> <line branch="false" hits="51" number="12"/>
> <line branch="false" hits="51" number="13"/>
> <line branch="false" hits="0" number="32"/>
> <line branch="false" hits="0" number="33"/>
> <line branch="false" hits="0" number="34"/>
> <line branch="false" hits="0" number="36"/>
> <line branch="false" hits="0" number="39"/>
> <line branch="false" hits="0" number="40"/>
> <line branch="false" hits="0" number="41"/>
> <line branch="false" hits="0" number="56"/>
> <line branch="false" hits="0" number="69"/>
> <line branch="false" hits="0" number="84"/>
> </lines>
> </class>
> </classes>
> </package>
> </packages>
> </coverage>
> 
> ```

and here is the CI job I use:

> **CI job**
>
> ```yml
> .linux_test_coverage:
> rules:
> - !reference [.test_rules, rules]
> extends:
> - .linux_before
> coverage: /Test coverage (\d+\.\d+%)/
> script:
> # First we add registries and the LocalCoverage package in the main env
> - |
> julia -e '
> using Pkg
> Pkg.Registry.add("General")
> Pkg.add("LocalCoverage")'
> # Then we create the coverage report and percentage
> - |
> julia --project=@. -e '
> using Pkg
> Pkg.build()
> using LocalCoverage
> cov = generate_coverage()
> c, t = cov.lines_hit, cov.lines_tracked
> using Printf
> # This printing is needed to extract the coverage percentage from the pipeline to show on the README badge
> @printf "Test coverage %.2f%%\n" 100c / t
> show(cov) # This will print the coverage table on the job log
> write_lcov_to_xml("coverage/cobertura-coverage.xml", "coverage/lcov.info")'
> artifacts:
> reports:
> coverage_report:
> coverage_format: cobertura
> path: coverage/cobertura-coverage.xml
> 
> ```

The `.linux_before` just sets up the image by installing `git` and setting up git credentials, while the `.test_rules` one just specifies rules for triggering the job.

You probably know that, but `LocalCoverage` only tracks files that are located in the `src` folder (and its subfolders), so if you have source code that is in another folder coverage for those is not computed.

Also by default the coverage file artifacts expires after 1 week if I am not mistaken, but should not expire while the MR is still open.

Here is then an example of the coverage visualization I get:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/0/00078f157c06b06282d56af3b8eda39457bf8487.png)

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [July 11, 2023, 12:38pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/29 "2023-07-11T12:38:49Z")

</div>

Thanks for that. As far as I can tell, there is not obvious difference between our Cobertura files (I actually implemented the native conversion to cobertura in LocalCoverage). I’ll fiddle a bit with the .gitlab-ci files since these are pretty different in terms of content…

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [July 11, 2023, 2:41pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/30 "2023-07-11T14:41:31Z")

</div>

@disberd I’m wondering, what is the structure of your repo ? Directly the julia project directory with

```julia
- src/
- Project.toml

```

?

---

<div class="post-metadata">

**Author:** ![disberd](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@disberd](https://discourse.julialang.org/u/disberd)\
**Post date:** [July 11, 2023, 2:46pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/31 "2023-07-11T14:46:52Z")

</div>

Yes, all my projects have the package folder as root (so as you described above)

---

<div class="post-metadata">

**Author:** ![BambOoxX](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bambooxx/32/22179_2.png) [@BambOoxX](https://discourse.julialang.org/u/BambOoxX)\
**Post date:** [July 11, 2023, 2:49pm UTC](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685/32 "2023-07-11T14:49:16Z")

</div>

So that may explain why it does not work on my case… I’ll try to path my cobertura file as discussed in [Cobertura coverage not showing in diff or jobs - #5 by Btripp1986 - GitLab CI/CD - GitLab Forum](https://forum.gitlab.com/t/cobertura-coverage-not-showing-in-diff-or-jobs/54798/5)

[Previous page](https://discourse.julialang.org/t/julia-and-gitlab-self-hosted-a-state-of-the-art/86685.md?page=1)