How to submit code coverage results to codecov/coveralls via GitLab CI?

General Usage
, , ,
1

Hi everyone,

I’m working on developing my first Julia package, and I want to make sure I have the code coverage aspect down before committing to developing and hosting the package on GitLab (which I prefer for a few reasons). I’ve seen some examples of how to submit code coverage results to codecov and coveralls using the Travis CI in Github, but for my project, I’m planning to use GitLab’s built-in CI. Can I simply add

after_success:
  - julia -e 'cd(Pkg.dir("PkgName")); Pkg.add("Coverage"); using Coverage; Codecov.submit(Codecov.process_folder())'

to my .gitlab-ci.yml file just as I would to a .travis.yml file? (and add the codecov token to my environment variables in gitlab?)

Thanks for any help!

  • Vincent
2

I created a package recently and the .travis.yml of it works. I think you need to give Travis and Codecov permission to access your repository (something like that), and Travis has a web interface that shows you in real time the output of the build when you commit a new version (to help you debug why it is not working). I had never done that before and I have to admit that I suffered a little to make it working. Until I discovered the Travis web interface it was not working because the yml was wrongly indented and I did not know. My commit history shows in that file shows my suffering.

Also, you probably need import Pkg before using it.

3

I got an e-mail about an issue you opened at

but then I cannot find it in the list of issues. Weird.

In any case, I explore and add @Henrique_Becker’s solution to the template.

4

Are you sure this works on Gitlab? Your repo is on Github.

5

I am very sorry, I have read the e-mail when I was tired and did not notice that it was GitLab specific. My bad.

6

Thanks for the replies! Hmm, looks like the authors of the CuArrays.jl repo may have it figured out? It’s hosted on GitHub, but uses GitLab CI and publishes coverage to codecov.io. I’ll reach out to them and post back to this thread if they have a solution!

7

Looks like Coverage.jl doesn’t have support for direct integration with GitLab CI. I opened an issue here: https://github.com/JuliaCI/Coverage.jl/issues/226

8

Alright! I got it figured out. Turns out even though Coverage.jl doesn’t have support for GitLab, you can still publish results to codecov.io by adding this:

after_script:
  - julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit_local(process_folder())

to the .gitlab-ci.yml file, and making sure you’ve added an environment variable to your gitlab project under settings :arrow_forward: CI/CD :arrow_forward: variables with key “CODECOV_TOKEN” and value set to the repository upload token provided by codecov. Make sure you don’t put quotes around the value either!

@Tamas_Papp If you want I can fork your GitLab example repo and put in a merge request with the necessary additions (but you’ll need to add the codecov token).

5 Likes
9

A merge request would be appreciated. The instructions above should go in the README, too. I will of course add a token.

10

@Tamas_Papp I changed so much in my forked repo (and so lots would be changed if I merged it) that I think it might make the most sense for you to just update the yml file in your repo manually. Here’s the code:

image: julia:1.1                # image comes from Docker hub

before_script:
  # workaround for https://github.com/JuliaDocs/Documenter.jl/issues/686
  - apt-get -qq update; apt-get -y install git
  - julia --project=@. -e "import Pkg; Pkg.build()"

default:
  script:
    - julia --project=@. -e "import Pkg; Pkg.test(; coverage = true)"
    - julia --project=test/coverage -e 'import Pkg; Pkg.instantiate()'
    - julia --project=test/coverage test/coverage/coverage-summary.jl

pages:
  stage: deploy
  script:
    - julia --project=docs -e 'using Pkg; Pkg.instantiate(); Pkg.develop(PackageSpec(path=pwd()))'
    - julia --project=docs --color=yes docs/make.jl
    - mv docs/build public # move to the directory picked up by Gitlab pages
  artifacts:
    paths:
      - public
  only:
  - master

after_script:
  - julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit_local(process_folder())'

As you can see I only had to add the one line. You can find the badge URL in codecov under settings for the project.

2 Likes
11

It took a bit more than that, but now I fixed this in the repo. Thanks for investigating this.

1 Like
12

@Tamas_Papp Sure thing! What else did you have to do aside from adding the token and initializing the codecov project? Just want to cover all the bases in this thread so others can use it in the future.

13

You can see the MR here:

Basically, it would not run without the codecov file for me, and one does not need an additional script to submit, since we use the Project.toml-based framework as on Travis. This is also faster since the coverage info is calculated already.

14

Ah I see, you just added the coverage submission to the coverage script in test/coverage. In my case, I don’t have a coverage script at all since I’m just using codecov, so I stuck with using the after_script command in the yml. Interestingly I didn’t need a codecov.yml, but will keep it in mind in case I run into problems in the future. Thanks for your help with all of this!

1 Like