The @timed macro provides a bunch of very interesting stats about GC and compilation time.
I wanted to reuse this data to estimate the time excluding compilation and GC. However since (I believe) GC can and will happen during compilation, removing the gc_time and compilation_time will remove the time spent on GC during compilation twice.
Do you think there is a way to build a custom solution (some modified version of @timed) that would allow making the distinction?
I don’t think @timed separately measuring GC in compilation versus execution is hypothetically impossible, but it’ll significantly change and complicate an implementation already complicated by exceptions thrown from multiple threads and scoping. Maybe you can temporarily switch off GC.enable(false) to do a timing without GC, and you can subtract the compilation timing yourself. That’'ll risk memory problems if your call really needs GC, and one very fast call makes for very poor execution timing. Alternatively, initial compilation overhead usually becomes an outlier in repeated benchmark runs (which multiply the risk of switching off GC so don’t do that), so you can roughly subtract BenchmarkTools stats of GC from execution in the majority of cases.
I think if you build Julia with support for Tracy, and then trace the Julia process with it, you’ll get that info and more:
Thanks for the pointer! That sounds like an interesting approach, I am a bit confused by the explanation in the docs, but I will investigate it further.
I would also like to disable GC unfortunately the tests I run are pretty memory hungry and it’s complicated to run them in a benchmark (too many of them). But thanks for the suggestion!