Benchmarking function in local scope

General Usage
, ,
1

I have a function that I would like to benchmark. To that end, I would like to time the function for several different argument values, and plot how the runtime changes with the argument values. I can do this using the @elapsed macro, but to get a more accurate/reliable result I would like to use the @belapsed macro from BenchmarkTools. E.g.

julia> using BenchmarkTools

julia> function f()
         a = "Hello World"
         t = @belapsed println(a)
       end
f (generic function with 1 method)

julia> f()
ERROR: UndefVarError: `a` not defined

I cannot use @belapsed because the function I am benchmarking uses variables in the local scope of the function it is called within. According to the BenchmarkTools documentation: " @benchmark is evaluated in global scope, even if called from local scope." I guess the same is true of @belapsed.

This is a surprising design choice to me, but I don’t see a workaround besides executing the body of the outer function in the global scope, which would be very inconvenient.

Does anyone know a workaround or another package to use so I can get a reliable estimate of the runtime for a function, even when called inside the scope of another function?

2

Try escaping using println($a).

3 Likes
3

That works, thanks a bunch! I did not know that the $ operator could be used like that. May I ask, what exactly does $ do outside of a string? From the REPL, if I just do $a for some variable a, I get an error:

julia> $a
ERROR: syntax: "$" expression outside quote around REPL[103]:1

And the help string for $ doesn’t help me much:

help?> $
search: $

  $

  Interpolation operator for interpolating into e.g. strings and expressions.

  Examples
  ≡≡≡≡≡≡≡≡

  julia> name = "Joe"
  "Joe"
  
  julia> "My name is $name."
  "My name is Joe."
4

$ is a feature of the BenchmarkTools.jl macros. See the docs for more details.

If the expression you want to benchmark depends on external variables, you should use $ to “interpolate” them into the benchmark expression to avoid the problems of benchmarking with globals. Essentially, any interpolated variable $x or expression $(...) is “pre-computed” before benchmarking begins

https://juliaci.github.io/BenchmarkTools.jl/stable/

1 Like
5

Awesome, thanks!