Enabling code coverage changes how Julia code is compiled and executed. In particular, code coverage may cause allocations in code that would otherwise be constant-folded:
On the other hand, it’s sometimes good to have tests in a package’s testsuite to check whether certain calls allocate or not. It seems these tests need to be disabled while coverage is enabled.
But how to detect whether coverage is enabled?
Taking my question of “how to detect code coverage” literally, the answer is that there’s Base.JLOptions.code_coverage (not public). However, I realize that I was asking the wrong question, e.g., what if the user decides to disable compilation, or lower the optimization level; that could presumably cause allocation just like code coverage did. But testing for each compiler option that may result in suboptimal performance isn’t robust, because I could never be sure I covered all the possible options.
A better question to ask might be “how to detect a production build”, because for a production build I can tell with certainty that some calls mustn’t allocate. I’ll change the title.
I guess it’d be fine if there was an additional compiler flag, which would convey whether the intent of the build is “production” or not. I’d also need a public API to query the value of this flag, perhaps a Preferences.jl preference would suffice. Then I could use this in the test suite to enable/disable certain tests. I’ll make an issue for the FR on Github. EDIT: actually, this seems more like it should be a package.
By whom is it intended?
If a “production” build mode is available, users will always expect a “production” build mode as the default.
Some users want stability over speed in their products. Some users want detailed logging.
In the end, if how the flags are used is package-dependent, it is something that should be managed by the package.
By the user, which would most often be CI, in this case, but also might be me if I decide to run the test suite locally.
Anyway, as someone suggested in the other thread, using an environment variable seems like the best approach.