# Benchmark is moving target?

**URL:** <https://discourse.julialang.org/t/benchmark-is-moving-target/123376>\
**Category:** Performance\
**Created:** [December 3, 2024, 1:12am UTC](https://discourse.julialang.org/t/benchmark-is-moving-target/123376 "2024-12-03T01:12:59Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [December 3, 2024, 1:12am UTC](https://discourse.julialang.org/t/benchmark-is-moving-target/123376/1 "2024-12-03T01:12:59Z")

</div>

I am doing Advent of Code again this year (not sure how long I’d stay haha) and tried to optimize a very simple function. Surprisingly, `@btime` is showing different number of allocations on 2nd run. Also, when I redefined my function then the number of allocations dropped to zero. What’s going on? Can I still trust BenchmarkTools?

Source: [AdventOfCode2024/day1.jl at main · tk3369/AdventOfCode2024 · GitHub](https://github.com/tk3369/AdventOfCode2024/blob/main/day1.jl)

My REPL:

```julia
julia> include("day1.jl");

julia> @btime part2_fast($x, $y)
  139.041 μs (2 allocations: 32 bytes)
21024792

julia> @btime part2_fast($x, $y)
  139.000 μs (1 allocation: 16 bytes)
21024792

julia> part2_fast(x, y) = sum(value * count(==(value), y) for value in x)
part2_fast (generic function with 1 method)

julia> @btime part2_fast($x, $y)
  139.000 μs (0 allocations: 0 bytes)
21024792

```

---

<div class="post-metadata">

**Author:** ![barucden](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barucden/32/26154_2.png) [@barucden](https://discourse.julialang.org/u/barucden)\
**Post date:** [December 3, 2024, 9:43am UTC](https://discourse.julialang.org/t/benchmark-is-moving-target/123376/2 "2024-12-03T09:43:48Z")

</div>

It seems it has something to do with the generator inside the `sum` argument. The allocations go away reliably when I switch the `sum` method:

```julia
f(x, y) = sum(v -> v * count(==(v), y), x)

```

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [December 3, 2024, 10:06am UTC](https://discourse.julialang.org/t/benchmark-is-moving-target/123376/3 "2024-12-03T10:06:07Z")

</div>

you can plot the distrubition of times using `@benchmark` instead of `@btime`