Variance in @elapsed while trying to measure function

Hello, good afternoon. Im using @elapsed to mesure time function and got some flutuation in sequence data. Is this peaks in plot gargabe collection noise?

Good evening! It might be gc as you say. Consider using BenchmarkTools.jl for accurate benchmarking. E.g., using @btime or @belapsed

2 Likes

Im testing here but it is taking a lot of time. Seems @belapsed cant run with more than 1000 iterations.

The functions with constant time was done, but seems more strange than using @elapsed

You can control the number of evaluations BenchmarkTools.jl is using. The timing that is less than one nanosecond means that the compiler outsmarted the benchmark and performed the entire calculation at compile time. This is indicated on the first page of the docs Home · BenchmarkTools.jl

As a rule of thumb, if a benchmark reports that it took less than a nanosecond to perform, this hoisting probably occured. You can avoid this by referencing and dereferencing the interpolated variables

2 Likes

Both @belapsed and @btime already run your code more than once to collect statistics from various runs. There’s no need to invoke them in a loop.

2 Likes

But how to plot a performance curve with only one value collected?

Have you checked out the documentation of BenchmarkTools.jl? You can use @benchmark to get an object containing the collected statistics.

1 Like