Pluto and macros

New to Julia
1

I need to benchmark a function and I decided to use Pluto. I get a variable not defined error that I don’t know how to avoid. A minimal (not) working example is:

for i in 1:length(ar)
	b = @benchmark i^2
end

Is there any way to use the benchmark macro in a loop in Pluto or should I just write a script and run it?

2

Did you try this outside of Pluto? I don’t think this has anything to do with Pluto.

@benchmark will actually involve using eval, so it doesn’t know about locally scoped variables like i here. However, you can interpolate those variables into the benchmarked expression like this:

for i in 1:length(ar)
	b = @benchmark ($i)^2
end

Now however, julia’s compiler is being given too much information and is totally optimizing this away:

julia> let i = 10
           @benchmark ($i)^2
       end
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.019 ns (0.00% GC)
  median time:      0.020 ns (0.00% GC)
  mean time:        0.024 ns (0.00% GC)
  maximum time:     0.120 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

Whenever you see a benchmark that takes less than a nanosecond, that’s a strong indication that nothing actually happened in the benchmark loop.

In order to actually benchmark what you want to do, you have to block the compiler from seeing i as a constant. One way to do that would be to wrap it in a mutable Ref like this:

julia> let i = Ref(10)
           @benchmark ($i[])^2
       end
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     1.329 ns (0.00% GC)
  median time:      1.340 ns (0.00% GC)
  mean time:        1.360 ns (0.00% GC)
  maximum time:     157.327 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

Now that is properly benchmarking the thing I think you want to measure.

3 Likes
3

I tried it in the mean time in a script and I got the same error. :frowning:

How do I use Ref? Do I just write

xr = Ref(x)
@benchmark f($xr[])

instead of

@benchmark f(x)

?

1 Like