Code coverage works poorly in Julia 0.7

As we upgraded from Julia 0.6 to Julia 0.7, our code test coverage jumped from 73% to 87%. Looking more closely at the coverage report, it is now very inaccurate. Lines are missed, or counted double. Entire functions and files that are not run are not included at all in the report.

I’ve looked at coverage reports for other Julia 0.7 and 1.0 projects, and it seems the same everywhere.

Even with --inline==no I get incorrect results.

There’s a github issue, but it doesn’t seem to attract a lot of attention.

Did anyone else reflect over this? Thoughts? Workarounds?

I noticed similar things. I get 100% coverage reports on files that are only loaded with requires or never actually loaded at all.

Don’t trust the green lights.