Does Julia have, or would it be possible to add, a compilation-ignoring timer?
Use case: setting a time limit for running a function, as is common for optimizers. The first execution will be necessarily longer than the later executions due to compilation, but you may want to set a time limit that’s a lot shorter than that first execution.
A workaround is to ensure it is compiled prior to optimization, but that’s a bit annoying to remember to do each time.
A bit easier would be if packages that need to track time could do so in a way that tries to ignores compilation time. And for that, you’d need a timer that can be informed that Julia is currently compiling.
I’m guessing this isn’t implemented because it wouldn’t be as useful as it seems: even if you don’t count the compilation time towards the measured time, the compilation would still interfere with the performance so the resulting timings wouldn’t be completely relevant.
EDIT: On the other hand, it’s weird that we have
@timev which tells us how much of the total elapsed time was spent compiling, but
@timed doesn’t give us that data.
GitHub - JuliaCI/BenchmarkTools.jl: A benchmarking framework for the Julia language is the defacto standard package for this.
This package is for interactive benchmarking, which is not what I’m asking for. All it does to “ignore compilation” is to pre-run the method.
To the first, I don’t think an imperfect adjustment is bad. If more could be done to account for other first-run delays, that would be great.