Benchmarking a function with side effects

General Usage
,
1

I want to compare push!ing an element to an Array with push!!ing an element to a Tuple using BenchmarkTools.jl and BangBang.jl:

julia> using BenchmarkTools, BangBang

julia> a = 1:20 |> collect;

julia> b = Tuple(a);

julia> @btime push!(a, 1)
  19.274 ns (0 allocations: 0 bytes)
10470522-element Array{Int64,1}:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  ⋮
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1
  1

julia> @btime push!!(b, 1)
  26.583 ns (1 allocation: 176 bytes)
(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 1)

However, push! will change a within each loop of the benchmark while push!! will not change the Tuple. That forms unfair comparisons (even though in this case, push! seems to be much faster?). How can I benchmark pushing exactly 1 element into a?

2

Use setup

@benchmark push!(a,1) setup=(a=collect(1:20))
2 Likes
3

Also evals=1 to avoid repeatedly pushing to the same array (ref: How to benchmark append!? - #7 by rdeits )

7 Likes