# Pluto and macros

I need to benchmark a function and I decided to use Pluto. I get a variable not defined error that I don’t know how to avoid. A minimal (not) working example is:

``````for i in 1:length(ar)
b = @benchmark i^2
end
``````

Is there any way to use the benchmark macro in a loop in Pluto or should I just write a script and run it?

Did you try this outside of Pluto? I don’t think this has anything to do with Pluto.

`@benchmark` will actually involve using `eval`, so it doesn’t know about locally scoped variables like `i` here. However, you can interpolate those variables into the benchmarked expression like this:

``````for i in 1:length(ar)
b = @benchmark (\$i)^2
end
``````

Now however, julia’s compiler is being given too much information and is totally optimizing this away:

``````julia> let i = 10
@benchmark (\$i)^2
end
BenchmarkTools.Trial:
memory estimate:  0 bytes
allocs estimate:  0
--------------
minimum time:     0.019 ns (0.00% GC)
median time:      0.020 ns (0.00% GC)
mean time:        0.024 ns (0.00% GC)
maximum time:     0.120 ns (0.00% GC)
--------------
samples:          10000
evals/sample:     1000
``````

Whenever you see a benchmark that takes less than a nanosecond, that’s a strong indication that nothing actually happened in the benchmark loop.

In order to actually benchmark what you want to do, you have to block the compiler from seeing `i` as a constant. One way to do that would be to wrap it in a mutable `Ref` like this:

``````julia> let i = Ref(10)
@benchmark (\$i[])^2
end
BenchmarkTools.Trial:
memory estimate:  0 bytes
allocs estimate:  0
--------------
minimum time:     1.329 ns (0.00% GC)
median time:      1.340 ns (0.00% GC)
mean time:        1.360 ns (0.00% GC)
maximum time:     157.327 ns (0.00% GC)
--------------
samples:          10000
evals/sample:     1000
``````

Now that is properly benchmarking the thing I think you want to measure.

3 Likes

I tried it in the mean time in a script and I got the same error.

How do I use `Ref`? Do I just write

``````xr = Ref(x)
@benchmark f(\$xr[])
``````

``````@benchmark f(x)