Benchmark in place functions with @btime wasn't set up properly?

General Usage Performance
1

I wrote an in-place function that performs the multidimensional Newton-Raphson algorithm, and I’m trying to compare the performance with the original implementation in Matlab.

Since I wrote it as an in-place function, I used
@btime nr!($data, $options) setup(data=load_f("file.m")) evals=1
to force it to reload a new data variable from a local file each time, so that updating the data with solved results doesn’t affect the next run.

This results in an almost 12x speedup compared to the original Matlab code, which I was pretty excited about at first, but the result is a little worrying: nr! returns the mismatch at each NR iteration as well as the number of iterations used to reach the final acceptable mismatch. When I simply did

data = load_f("file.m")
nr!(data, options)  # options contains tolerance, max iterations, etc. defined elsewhere

it returns the mismatches and iterations (3) as expected. But with @btime and the setup, evals I got just the final mismatch from before and 0 iteration, which seems to me that the setup wasn’t done properly and it was running with a data with already solved values.

I also tried to do the following and received similar results.

data = load_f("file.m")
@btime nr!($data_copy, $options) setup(data_copy=deepcopy(data)) evals=1

Could anyone help me understand what’s going on here? What am I doing wrong?

Thanks very much!

2

My solution to this (not the most elegant) is to just use @time several times (and ignore the first). Not the best answer, but I know it will do what I want.

3

Yes it works, and the result of @time actually shows only about 2x speedup compared to the Matlab implementation (LOL but more like what I expected). I still want to use @btime if possible though, because even if I get away with just @time this time doesn’t mean I can do it in future…

Also, I commented the in-place section of nr! out, but interestingly, there’s still quite a bit of discrepancy between @btime (610 us) and @time (920 us), and the result recorded from

function nr!(data, options)
    ts = time()
    newton raphson body 
    time() - ts
end

is closer to @time than @btime and actually even larger than @time’s result. I’m so confused…