# Benchmarking the cost of allocation

**URL:** <https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435>\
**Category:** General Usage\
**Tags:** question\
**Created:** [October 29, 2019, 9:42am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435 "2019-10-29T09:42:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 29, 2019, 9:42am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/1 "2019-10-29T09:42:50Z")

</div>

I would like to have an educated guess of how much speed I would gain if I rewrote a rather large codebase with preallocated buffers (all the other low-hanging fruits I could think of have been harvested at this point, eg checking type stability, profiling, fixing hot loops, static arrays, etc).

Essentially, I would like to measure just the _cost_ of allocations, separately from the rest of the computation. I can of course count allocations with `BenchmarkTools` and `@time`, but I don’t know how to translate that into performance.

Conceptually, I would like to do something like the comparison

```julia
using BenchmarkTools
f(a, b) = map((a, b) -> √abs(a) + exp(b), a, b)
f!(result, a, b) = map!((a, b) -> √abs(a) + exp(b), result, a, b)
a = randn(1000)
b = randn(1000)
result = similar(a)
@btime f($a, $b);
@btime f!($result, $a, $b);

```

**without writing `f!`**.

Any hints would be appreciated.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 29, 2019, 9:52am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/2 "2019-10-29T09:52:11Z")

</div>

Time spent in GC is some sort of measurement (reported by `@benchmark`).

Accurate prediction seems extremely hard since even if you measure the time spent allocating and running GC it is hard to predict the other effects this has on the system (like how it influences cache etc).

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [October 29, 2019, 10:53am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/3 "2019-10-29T10:53:46Z")

</div>

Yeah, I often find the speedup quite a lot greater in practice than what I would have guessed based on the time previously spent on gc.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 29, 2019, 11:08am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/4 "2019-10-29T11:08:55Z")

</div>

Yes, in the example above GC time is 0 for me (for `f`).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 29, 2019, 11:52am UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/5 "2019-10-29T11:52:36Z")

</div>

Expanding on this: I get

```julia
julia> @benchmark f($a, $b)
BenchmarkTools.Trial: 
  memory estimate: 8.00 KiB
  allocs estimate: 4
  --------------
  minimum time: 12.665 μs (0.00% GC)
  median time: 15.040 μs (0.00% GC)
  mean time: 15.395 μs (0.00% GC)
  maximum time: 186.778 μs (0.00% GC)
  --------------
  samples: 10000
  evals/sample: 1

```

I wonder why GC time is consistently zero. Should this be reported as an issue for BenchmarkTools or Julia?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 29, 2019, 12:09pm UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/6 "2019-10-29T12:09:07Z")

</div>

BenchmarkTools runs GC itself between samples so that might take care of all the GC needed.

You could try set

```julia
BenchmarkTools.DEFAULT_PARAMETERS.gctrial = false

```

and see if anything changes.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 29, 2019, 12:17pm UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/7 "2019-10-29T12:17:25Z")

</div>

Thanks, but that didn’t change anything. Neither did

```julia
BenchmarkTools.DEFAULT_PARAMETERS.samples = 1_000_000

```

My I understanding is that allocating dynamically is costly because

1. of allocation (creating a new array etc)
2. GC time

The way I understand this discussion is that currently it is conceptually difficult to measure (1), while for (2) we have facilities but I am not able to trigger them in practice.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 29, 2019, 12:21pm UTC](https://discourse.julialang.org/t/benchmarking-the-cost-of-allocation/30435/8 "2019-10-29T12:21:30Z")

</div>

I get

```julia
julia> @benchmark f($a, $b)
BenchmarkTools.Trial:
  memory estimate: 8.00 KiB
  allocs estimate: 4
  --------------
  minimum time: 8.092 μs (0.00% GC)
  median time: 8.289 μs (0.00% GC)
  mean time: 8.581 μs (1.87% GC)
  maximum time: 9.631 ms (99.85% GC)
  --------------
  samples: 188641
  evals/sample: 3

```

so for me, some GC time is shown.

Your maximum time is quite low so it just doesn’t seem that any GC was triggered when measuring.