# Why do \`$\`s in BenchmarkTools change allocations so dramatically?

**URL:** <https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779>\
**Category:** General Usage\
**Tags:** memory-allocation\
**Created:** [January 18, 2022, 1:56am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779 "2022-01-18T01:56:47Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 1:56am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/1 "2022-01-18T01:56:48Z")

</div>

**EDIT: PLEASE IGNORE THIS ENTIRE THREAD.**  
I will start another one.

Weird. I have a copy of vectors.

```julia
using LinearAlgebra
using BenchmarkTools

N = 10
U1 = rand(N)
U0 = rand(N)

@btime copyto!(U1, U0)  
@btime $U1 .= $U0  
@btime @. U1 = U0  

```

The output is:

```julia
  19.238 ns (0 allocations: 0 bytes)                                                   
  7.407 ns (0 allocations: 0 bytes)                                                    
  254.743 ns (2 allocations: 32 bytes)   

```

I don’t know the answer to these questions. Do you?

1. In the first and second case there is no allocation. Why is the first operation more than twice as  
expensive?

2. The difference between the second and third case is just the quoting. Why does the third case allocate? And why 32 bytes?

Edit: With Julia 1.7.1. Haven’t tried 1.6.5 yet.  
Edit 2: Same results with 1.6.5.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [January 18, 2022, 4:17am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/2 "2022-01-18T04:17:18Z")

</div>

I guess, you forgot the interpolation (`$` signs):

```julia
@btime copyto!($U1, $U0)  
@btime $U1 .= $U0  
@btime @. $U1 = $U0

```

```julia
5.749 ns (0 allocations: 0 bytes)
7.479 ns (0 allocations: 0 bytes)
7.266 ns (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 4:55am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/3 "2022-01-18T04:55:28Z")

</div>

Sorry, I said quotation. I meant interpolation. And that was the point. No interpolation: why allocation? And a huge hit in performance.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 4:56am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/4 "2022-01-18T04:56:07Z")

</div>

And why is your `copyto!` so much faster? Oh, I see interpolation again? But why?

```julia
@btime copyto!(U1, U0)
@btime copyto!($U1, $U0)  
@btime $U1 .= $U0
@btime @. U1 = U0

```

gives

```julia
  26.533 ns (0 allocations: 0 bytes)
  8.408 ns (0 allocations: 0 bytes)
  11.712 ns (0 allocations: 0 bytes)
  342.593 ns (2 allocations: 32 bytes)

```

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 5:03am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/5 "2022-01-18T05:03:52Z")

</div>

Ah, I think I forgot about the “global” character of variables not interpolated.  
But still: I am passing them into a function, where they will be happily local.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 5:17am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/6 "2022-01-18T05:17:57Z")

</div>

Sorry, I was trying to construct a MWE of a puzzling situation, but I don’t think I managed to capture it with this. I will try again later.

---

<div class="post-metadata">

**Author:** ![fedoroff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fedoroff/32/53209_2.png) [@fedoroff](https://discourse.julialang.org/u/fedoroff)\
**Post date:** [January 18, 2022, 6:18am UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/7 "2022-01-18T06:18:35Z")

</div>

> [@PetrKryslUCSD](#):
>
> I am passing them into a function, where they will be happily local

Maybe the machinery of the `@btime` macro itself uses these variables.  
They have some additional information in the readme for the BenchmarkTools:  
[https://github.com/JuliaCI/BenchmarkTools.jl#benchmarktoolsjl](https://github.com/JuliaCI/BenchmarkTools.jl#benchmarktoolsjl)

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 2:57pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/8 "2022-01-18T14:57:06Z")

</div>

The example in the readme does not quite explain the effect of using a global variable:

```julia
using LinearAlgebra
using BenchmarkTools

A = rand(3, 3);

@btime inv($A);
@btime inv(A);
@btime inv($(rand(3, 3)));
@btime inv(rand(3, 3));

nothing

```

Yields

```julia
  722.656 ns (4 allocations: 1.86 KiB)
  746.087 ns (4 allocations: 1.86 KiB)
  723.438 ns (4 allocations: 1.86 KiB)
  871.429 ns (5 allocations: 1.98 KiB)

```

So, not much of a difference between `A` interpolated or not. So, why the huge difference for `copyto!`?

Edit: For larger vectors the difference between `@btime copyto!(U1, U0)` and `@btime copyto!($U1, $U0)` disappears. Still: why is there a difference for small vectors?

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 4:07pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/9 "2022-01-18T16:07:57Z")

</div>

The thing I am trying to wrap my head around is that I observed that a broadcast to combine three vectors would allocate in a loop:

```julia
@time for step in 1:nsteps
        @. U1 = U0 + dt*V0 + ((dt^2)/2)*A0; 
end

```

yields

```julia
  3.080494 seconds (130.00 k allocations: 4.883 MiB)    

```

The number of allocations grows linearly with the upper bound of the range (`nsteps`).  
The code is not global, the loop exists inside of function.

The problem is that this doesn’t show in a trivial code, only in my original code.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 4:32pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/10 "2022-01-18T16:32:48Z")

</div>

Recall that the loop refers to the variable `dt`.  
The trigger is this: If I compute `omega_max` from the solution of an eigenvalue problem, the loop would allocate.

```julia
    evals, evecs, nconv = eigs(K, M; nev=1, which=:LM, explicittransform=:none)
    @show omega_max = sqrt(evals[1])
    # omega_max = 2.76450e+06 
    @show dt = Float64(0.99* 2/omega_max)
    @show typeof(dt)

```

On the other hand, if I set `omega_max` to a fixed value, the loop does not allocate.

```julia
    evals, evecs, nconv = eigs(K, M; nev=1, which=:LM, explicittransform=:none)
    # @show omega_max = sqrt(evals[1])
    omega_max = 2.76450e+06 
    @show dt = Float64(0.99* 2/omega_max)
    @show typeof(dt)

```

Did I mention that it was weird?

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 4:58pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/12 "2022-01-18T16:58:21Z")

</div>

All the code above is actually inside of a function. There is no global code.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [January 18, 2022, 6:32pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/14 "2022-01-18T18:32:11Z")

</div>

So I know you’ve found this thread to be a red-herring on your way to the [more complicated issue](https://discourse.julialang.org/t/this-is-some-serious-magic/74819/) you’re trying to debug, but there is a pretty straightforward answer to how BenchmarkTools uses that `$` syntax. It simply constructs its benchmarking loop such that anything that’s `$`'ed is an _argument_ to the function. Everything else is considered a global. How big of an effect this has is entirely dependent upon what Julia is doing.

In general, you want to make sure all _function-local variables_ in your original example have `$`'s on them in their benchmarks.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 7:28pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/15 "2022-01-18T19:28:00Z")

</div>

Still, `copyto!` is a function called with arguments that are known to the compiler. Whether it is interpolated or not.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 18, 2022, 7:37pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/16 "2022-01-18T19:37:42Z")

</div>

> [@PetrKryslUCSD](#):
>
> Still, `copyto!` is a function called with arguments that are known to the compiler. Whether it is interpolated or not.

The arguments are globals. The compiler knows they are globals. It has no way of knowing whether those globals are mutated inside `copyto!` or some ancillary function. Therefore each time it calls `copyto!` in the benchmark loop it must check the type of the global inputs and dispatch dynamically.

Interpolating tells the compiler the _value_ of the input, rather than the global variable that points to that value.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [January 18, 2022, 7:38pm UTC](https://discourse.julialang.org/t/why-do-s-in-benchmarktools-change-allocations-so-dramatically/74779/17 "2022-01-18T19:38:27Z")

</div>

Ah, _that_ makes sense. Ta.