# @benchmark reporting (probably) wrong memory usage (BenchmarkTools.jl)

**URL:** <https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301>\
**Category:** Tooling\
**Created:** [May 31, 2018, 7:19pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301 "2018-05-31T19:19:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![digital\_carver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digital_carver/32/33818_2.png) [@digital\_carver](https://discourse.julialang.org/u/digital_carver)\
**Post date:** [May 31, 2018, 7:19pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301/1 "2018-05-31T19:19:44Z")

</div>

(I’m asking this here instead of on [GitHub - JuliaCI/BenchmarkTools.jl: A benchmarking framework for the Julia language](https://github.com/JuliaCI/BenchmarkTools.jl) because it might just be my lack of understanding of what’s happening, and not an actual issue. If this doesn’t fit in here, let me know and I’ll open it as an issue there instead.)

I wrote some code (to solve a Project Euler problem) that seemed fairly straightforward and without any obvious big memory allocations:

```julia
function sum_digit_factorial_nums()
    result = 0
    for n in 10:9999999
        if digit_factorial_sum(n) == n
            result += n
        end
    end
    result
end

function digit_factorial_sum(n::Integer)
    nd = digits(n)
    return sum((factorial(d) for d in nd))
end

```

It runs reasonably fast (3-4 s), but the memory estimate by `@benchmark` seems way off:

```julia
BenchmarkTools.Trial:
  memory estimate: 1.68 GiB
  allocs estimate: 33721367
  --------------
  minimum time: 4.421 s (2.97% GC)
  median time: 4.427 s (2.91% GC)
  mean time: 4.427 s (2.91% GC)
  maximum time: 4.434 s (2.86% GC)
  --------------
  samples: 2
  evals/sample: 1

```

(this from Julia v0.6.2, the output is similar on v0.7 except slightly lower times and 1.48 GiB allocated instead of 1.68. Inlining the `digit_factorial_sum` function’s code made no real difference.)

The Julia process’ memory consumption did not raise anywhere near those levels during the execution, and there was no other indication that there was high memory usage going on (and believe me, I’d have felt _something_ on my Jurassic-era laptop!).

How should I interpret the memory estimate output here, and is this business as usual or is it a big enough discrepancy in the estimate to need an issue filed?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 31, 2018, 7:29pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301/2 "2018-05-31T19:29:25Z")

</div>

This is the total amount allocated, not the peak memory usage.

---

<div class="post-metadata">

**Author:** ![digital\_carver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digital_carver/32/33818_2.png) [@digital\_carver](https://discourse.julialang.org/u/digital_carver)\
**Post date:** [May 31, 2018, 7:36pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301/3 "2018-05-31T19:36:47Z")

</div>

Thanks. That was one explanation that came to mind, but I thought that information was (kinda) given by the allocs estimate, so this must be a different piece of information.

So if A and B are allocated and then go out of scope (and ready to be gc’d), then C and D are allocated, the memory estimate would be the sum of the memory usages of A, B, C, and D all? Is it fair to say the memory estimate shown is basically the sum of all `malloc`s done during the execution of the code?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 31, 2018, 7:43pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301/4 "2018-05-31T19:43:27Z")

</div>

You can look at the source: [https://github.com/JuliaCI/BenchmarkTools.jl/blob/master/src/execution.jl#L317-L336](https://github.com/JuliaCI/BenchmarkTools.jl/blob/master/src/execution.jl#L317-L336)

Before execution

```julia
__gc_start = Base.gc_num()

```

then after execution

```julia
__gcdiff = Base.GC_Diff(Base.gc_num(),__gc_start)

```

and the memory estimate is computed as

```julia
__memory = Int(fld(__gcdiff.allocd, __evals))

```

So it is the sum of all allocations during the execution of the codes.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [June 2, 2018, 5:33pm UTC](https://discourse.julialang.org/t/benchmark-reporting-probably-wrong-memory-usage-benchmarktools-jl/11301/5 "2018-06-02T17:33:08Z")

</div>

I believe that digits allocates.