Is GitHub-only code coverage possible?

Tooling
1

Currently if you want to get code coverage report, you need to register on Codecov / Coveralls website and then authorize the service to access your repositories. Those services are essentially third-party to GitHub.

So I’m looking for a GitHub Actions-based alternative to Codecov / Coveralls. Something akin to how Travis can be replaced with GitHub Actions.

Of course I expect it to support Julia, but I’m wondering does it exist in the first place?

2 Likes
2

The coverage data are generated by Julia itself.

julia --code-coverage  ....

So what these external services do are to get these data and show it in a human readable manner. Coverage.jl does a lot with these data. Probably you can set a simple action to generate a file tree vs coverage report.

5 Likes
3

Thanks for clarifying this. Now my next question is whether someone came up with a Julia package / GitHub workflow / GitHub app that does what Codecov / Coveralls do, i.e. make a badge and comment on PRs that change coverage?

4

For public repositories, when run via github actions, codecov does not need a token or other special access, but you do need to still run some third party code (their github action). You can audit it though, at least. See https://github.com/ericphanson/GuessworkQuantumSideInfo.jl/blob/4f20773aa564cc4d54bfb3be5e1e4a93c5a740fe/.github/workflows/ci.yml#L53 for an example. PkgTemplates should also configure this when you use Codecov and GitHub actions (https://github.com/invenia/PkgTemplates.jl/pull/181).

1 Like
5

That part is not problematic, but does it still require you to have a Codecov account and authorize it to access GitHub data?

6

I thought you did not need to authorize it to access your GitHub account with the new github action, but I haven’t actually tested that myself (since I already had authorized them from the previous way of doing it), so I am not totally sure. Likewise I’m not sure if you need to create a codecov account or not, but that seems less of an issue as well.

7

Yeah it is possible. I’ve got it working with ReportGenerator you can create report and deploy via github pages.

See JSONPointer.jl and XLSXasJSON.jl for example

It seems having coverage history graph is possible as well, but I didn’t get it working yet.

1 Like