# PSA: new version of codecov-action requires additional setup

**URL:** <https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857>\
**Category:** General Usage\
**Tags:** announcement, codecov, github-actions\
**Created:** [February 7, 2024, 8:50am UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857 "2024-02-07T08:50:01Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 7, 2024, 8:50am UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/1 "2024-02-07T08:50:01Z")

</div>

## What’s going on

[Codecov](https://about.codecov.io/) is a very useful tool to assess code coverage, that is, how much of the package source code is hit by the test suite. In GitHub continuous integration workflows, this happens through the [codecov-action](https://github.com/codecov/codecov-action), which recently released a [breaking version 4](https://github.com/codecov/codecov-action/releases/tag/v4.0.0). As a result, many package maintainers are receiving pull requests from [Dependabot](https://github.com/dependabot) to update their continuous integration `CI.yml` files., with changes like the following:

```yml
codecov/codecov-action@v3 -> codecov/codecov-action@v4

```

If you just merge this without other changes, **your coverage stats will no longer update** and the README badge will show a wrong percentage. Indeed, uploading these stats to the Codecov web service now requires setting a repository or organization [secret](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions). As stated in the release notes:

> Tokenless uploading is unsupported. However, PRs made from forks to the upstream public repos will support tokenless (e.g. contributors to OS projects do not need the upstream repo’s Codecov token). This [doc](https://docs.codecov.com/docs/adding-the-codecov-token#github-actions) shows instructions on how to add the Codecov token.

## What you need to do

If you are a package maintainer, read the [Codecov documentation on tokens](https://docs.codecov.com/docs/adding-the-codecov-token) and follow the instructions there. For each individual repository, you need to:

1. Retrieve a token from the Codecov settings of that repository
2. Add this token as a secret in the GitHub settings of that repository. In the secret value box, don’t put `CODECOV_TOKEN = abcd1234`, just put `abcd1234`.
3. Update the testing part of the `CI.yml` file [like so](https://github.com/codecov/codecov-action?tab=readme-ov-file#usage):

```yml
- uses: codecov/codecov-action@v4
    with:
      # possibly other stuff
      token: ${{ secrets.CODECOV_TOKEN }}
      fail_ci_if_error: false # or true if you want CI to fail when Codecov fails

```

If you are an organization owner, part of the work can be mutualized:

1. The token can be retrieved in the organization settings on Codecov, using a link like [https://app.codecov.io/account/gh/MyJuliaOrg](https://app.codecov.io/account/gh/MyJuliaOrg)
2. The secret can be set in the organization settings on GitHub
3. Each package’s `CI.yml` still needs an individual update [sigh]

Possible issues:

- The setting `fail_ci_if_error: true` might lead to unexpected CI errors that are due to Codecov server status or other things beyond our control.

## Lessons learned

When you receive a Dependabot PR, check the release notes for breaking changes like this one. They are usually in a collapsed section of the PR. I know I didn’t use to, but now I will.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/4/d/4dd2d0286f5817962a026fc1439fa106a35361c2.png)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [February 7, 2024, 1:05pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/2 "2024-02-07T13:05:50Z")

</div>

Thank you @gdalle for sharing these instructions. Really helpful ❤

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [February 7, 2024, 1:25pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/3 "2024-02-07T13:25:24Z")

</div>

I tried unsuccessfully so far. Can anyone share a working repository?

In my [case](https://github.com/JuliaPhysics/ABCDMatrixOptics.jl/actions/runs/7815422156/job/21318684768) I get some upload errors now:

> **log**
>
> ```julia
> Run codecov/codecov-action@v4
> ==> linux OS detected
> https://cli.codecov.io/latest/linux/codecov.SHA256SUM
> ==> Running version latest
> ==> Running version v0.4.6
> ==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-commit'
> /home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-commit
> gpg: directory '/home/runner/.gnupg' created
> gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
> gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
> gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
> gpg: Total number processed: 1
> gpg: imported: 1
> 
> gpg: Signature made Fri Feb 2 14:15:33 2024 UTC
> gpg: using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
> gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the owner.
> Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C 62FF 806B B28A ED77 9869
> 
> ==> Uploader SHASUM verified (103bfefcc56f76473179e600b96eb8150b0f349ad94836b0f63f03ffac469ad7 codecov)
> info - 2024-02-07 13:18:34,767 -- ci service found: github-actions
> warning - 2024-02-07 13:18:34,770 -- No config file could be found. Ignoring config.
> info - 2024-02-07 13:18:34,997 -- Process Commit creating complete
> error - 2024-02-07 13:18:34,998 -- Commit creating failed: {"error": "Server Error (500)"}
> ==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-report'
> /home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov create-report
> info - 2024-02-07 13:18:35,704 -- ci service found: github-actions
> warning - 2024-02-07 13:18:35,707 -- No config file could be found. Ignoring config.
> info - 2024-02-07 13:18:35,943 -- Process Report creating complete
> error - 2024-02-07 13:18:35,944 -- Report creating failed: {"error": "Server Error (500)"}
> ==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov do-upload'
> /home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov do-upload -f lcov.info
> info - 2024-02-07 13:18:36,648 -- ci service found: github-actions
> warning - 2024-02-07 13:18:36,651 -- No config file could be found. Ignoring config.
> warning - 2024-02-07 13:18:36,658 -- xcrun is not installed or can't be found.
> warning - 2024-02-07 13:18:36,660 -- No gcov data found.
> warning - 2024-02-07 13:18:36,660 -- coverage.py is not installed or can't be found.
> info - 2024-02-07 13:18:36,672 -- Found 1 coverage files to upload
> info - 2024-02-07 13:18:36,672 -- > /home/runner/work/ABCDMatrixOptics.jl/ABCDMatrixOptics.jl/lcov.info
> info - 2024-02-07 13:18:36,877 -- Process Upload complete
> error - 2024-02-07 13:18:36,878 -- Upload failed: {"error": "Server Error (500)"}
> 
> ```

EDIT: [here](https://github.com/roflmaostc/Radonka.jl) the same config works. So it might be related to the JuliaPhysics org.

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [February 7, 2024, 2:31pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/4 "2024-02-07T14:31:53Z")

</div>

JuliaDSP is using it successfully. But I have heard other orgs are having some challenges. Are there some settings that need toggling, like some permissions?

The codecov website was also a bit unstable yesterday, perhaps because everyone is doing the same thing. I suppose it will settle down in a few days.

-viral

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [February 7, 2024, 2:45pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/5 "2024-02-07T14:45:43Z")

</div>

I can only speak for JuliaPhysics where all my repos fail currently. JuliaGraphs fails too according to @gdalle.

Maybe @giordano knows more?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [February 7, 2024, 3:01pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/6 "2024-02-07T15:01:38Z")

</div>

Yeah, adding to token for JuliaPhysics is in my TODO list.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [February 7, 2024, 4:23pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/7 "2024-02-07T16:23:23Z")

</div>

But if I use the private key for my repo? Shouldn’t that work?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 7, 2024, 5:05pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/8 "2024-02-07T17:05:44Z")

</div>

In case it affected someone, there was a typo (bracket issue) in the CI excerpt above, fixed now

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [February 7, 2024, 5:09pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/9 "2024-02-07T17:09:54Z")

</div>

I set up the org-wide key in JuliaPhysics, and upload is working in [[GHA] Switch to `codecov/codecov-action` by giordano · Pull Request #141 · JuliaPhysics/Measurements.jl · GitHub](https://github.com/JuliaPhysics/Measurements.jl/pull/141)

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [February 7, 2024, 5:22pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/10 "2024-02-07T17:22:22Z")

</div>

I’m still confused why a separate key would not work?

But how could I access the key?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [February 7, 2024, 5:27pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/11 "2024-02-07T17:27:57Z")

</div>

It’s working now after deleting the key you created, and automatically using the org-wide one: [Update CI.yml · JuliaPhysics/ABCDMatrixOptics.jl@0d678bf · GitHub](https://github.com/JuliaPhysics/ABCDMatrixOptics.jl/actions/runs/7815422156/job/21329640561#step:8:48). Sounds like it was wrong?

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [February 7, 2024, 6:45pm UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/12 "2024-02-07T18:45:46Z")

</div>

Wrt to the original announcement and subsequent org-level tokens being added, it’s also worth noting that the Codecov action v3 can’t use org-level tokens. You should therefore update your CI file along with or after the Codecov v4 update. If you change your CI file to use the org-level token while still using Codecov v3, you would encounter errors uploading the coverage reports.

Repo tokens work with either version.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 8, 2024, 7:55am UTC](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857/13 "2024-02-08T07:55:06Z")

</div>

I’m getting “Server error (500)” on all of my personal and organization repos, including Graphs.jl, both yesterday and today. This is regardless of the tests passing, cause I set `fail_ci_if_error: false`: see the [CI log](https://github.com/JuliaGraphs/Graphs.jl/actions/runs/7826337073/job/21352010540?pr=339).  
I’d appreciate any help figuring this out! I hope this is temporary but the [status page of Codecov.io](https://status.codecov.com/) is all green.

> <https://github.com/codecov/feedback/issues/268>
>
> \*\*Describe the bug\*\*
> On several of my repos (both personal and in organizations…), whenever I try to update \`codecov-action\` to \`v4\`, I get a "Server Error (500)" during CI. This is of course after retrieving the Codecov token and storing it as a repo or organization secret (I actually \[wrote a guide\](https://discourse.julialang.org/t/psa-new-version-of-codecov-action-requires-additional-setup/109857?u=gdalle) on how to do this for the Julia community), so I don't think it's because of a bad setup on my end.
> 
> \*\*Environment (please complete the following information):\*\*
> - Browser: Firefox but irrelevant
> - System: GitHub Actions CI
> 
> \*\*To Reproduce\*\*
> Steps to reproduce the behavior:
> Example failed CI run at https://github.com/gdalle/DifferentiationInterface.jl/actions/runs/7826478656/job/21352421412?pr=18
> 
> \*\*Expected behavior\*\*
> I expected the code coverage report to be uploaded and a comment to appear on the PR. Instead the CI fails.
> 
> \*\*Screenshots\*\*
> !\[image\](https://github.com/codecov/feedback/assets/22795598/48abaa58-1342-44ae-983f-d718508b8562)
> 
> \*\*Additional context\*\*
> I thought this might be a temporary server error but:
> \- the codecov status page https://status.codecov.com/ is all green
> \- it happened both yesterday and today
> \- it happened on several of my repos (every one I've tried sofar)
> \- it didn't happen to some of my colleagues on other repos

EDIT: it was because I stored the secret in the wrong way. In the GitHub box for the secret value, put only `abcd1234`, not `CODECOV_TOKEN=abcd1234`. I edited the original post