# Aggregate coverage from multiple GH actions jobs

**URL:** https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490
**Category:** Tooling
**Tags:** coverage, codecov, ci, github-actions
**Created:** [December 12, 2023, 10:43am UTC](https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490 "2023-12-12T10:43:23Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [December 12, 2023, 10:43am UTC](https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490/1 "2023-12-12T10:43:23Z")

</div>

I have a repo which contains a main package and a sub package located in `/lib`, for which [I run CI in parallel using multiple GH-actions jobs (groups)](https://github.com/JuliaControl/ControlSystems.jl/blob/master/.github/workflows/test.yml#L25C1-L27C31). This generally works well, except that it’s tricky to get coverage reports including coverage from all jobs.

I make use of

```julia
      - uses: julia-actions/julia-processcoverage@v1
        with:
          directories: src,lib/ControlSystemsBase/src

```

to ensure that both `src` and the subpackage located in `lib/ControlSystemsBase/src` are included in the coverage, but this does not quite do what I want. Each job reports code coverage to codecov independently, and thus only includes the coverage for the code that was run for this particular job. This has the effect that the last job to finish and report the coverage “wins” by somehow overwriting the report of the earlier job, and codecov will thus only consider the lines covered by the last finishing job.

What I think I need to do is to let each job somehow save the `.lcov` files or the processed result, and then wait with processing coverage until the coverage info from all jobs is available. Does anyone know how to make something like this (or any other approach to the problem) happen using Github actions?

---

<div class="post-metadata">

### Author: ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [December 12, 2023, 1:37pm UTC](https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490/2 "2023-12-12T13:37:11Z")

</div>

> [@baggepinnen](#):
>
> Each job reports code coverage to codecov independently, and thus only includes the coverage for the code that was run for this particular job. This has the effect that the last job to finish and report the coverage “wins” by somehow overwriting the report of the earlier job, and codecov will thus only consider the lines covered by the last finishing job.

Are you sure about that? I was under the impression that Codecov somohow merged all code coverage results…

However, even if this is the case, it’s not ideal because you then receive alerts from codecov after each submission, until the last job finishes and the coverage statistics (as seen by codecov) are complete. So in any case I’d be interested to know if there is a good way to handle such cases.

---

<div class="post-metadata">

### Author: ![ranocha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ranocha/32/35588_2.png) [@ranocha](https://discourse.julialang.org/u/ranocha)
#### Post date: [December 12, 2023, 2:47pm UTC](https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490/3 "2023-12-12T14:47:31Z")

</div>

Codecov should indeed be able to handle that. You can use `.codecov.yml` (e.g., [https://github.com/trixi-framework/Trixi.jl/blob/3d899bc3a301bdc40fc57afb95ea9e88ab10c459/.codecov.yml](https://github.com/trixi-framework/Trixi.jl/blob/3d899bc3a301bdc40fc57afb95ea9e88ab10c459/.codecov.yml)) to tell it to report results only after having received X reports.

For Coveralls, the situation is a bit different since we have seen some problems there. But you can use a manual workaround like [https://github.com/trixi-framework/Trixi.jl/blob/3d899bc3a301bdc40fc57afb95ea9e88ab10c459/.github/workflows/ci.yml#L138-L207](https://github.com/trixi-framework/Trixi.jl/blob/3d899bc3a301bdc40fc57afb95ea9e88ab10c459/.github/workflows/ci.yml#L138-L207)

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [December 12, 2023, 4:47pm UTC](https://discourse.julialang.org/t/aggregate-coverage-from-multiple-gh-actions-jobs/107490/4 "2023-12-12T16:47:08Z")

</div>

Thanks, that appears to have solved half of my problems! 🙏 The other half was solved by adding a forgotten `Pkg.test(..., coverage=true)`. The full set of changes I had to do in my case, for anyone finding this thread in the future, can be found here

> <https://github.com/JuliaControl/ControlSystems.jl/pull/907/files>
