Unexpected behaviour while benchmarking

Hi fine folks!

While investigating improvements for another topic i just posted about here I ran into some unexpected behaviour while benchmarking the code with BenchmarkTools.

I have a function rolling_optima which I want to benchmark while I try to improve its performance.

If I run (in the REPL)
@btime rr = rolling_optima($aaa, 45);
I consistently get about 14.8 ±0.2 μs

If I then in the same source file copy the whole function and paste it but only with another name rolling_optima_better() , then include the source file in the REPL again and then run

@btime rr = rolling_optima_better($aaa, 45);
I then consistently get about 15.8 ±0.2 μs,

The new function is 1 μs slower though it is the exact same code. If I rename the new function to the old name (and the old function to the new name) and then include the source again the original name still runs faster.

Is this something related to some kind of caching maybe? I will have to change my routines for improving functions it seems…

If you start from a fresh REPL and include your function and run it, which time do you get?

Also, see https://github.com/timholy/Revise.jl

1 Like