What does 0 allocations: #=nonzero=# bytes mean?

General Usage
1

Sometimes BenchmarkTools reports 0 allocations but the number of bytes isn’t 0. It can vary between runs. What does that mean?

julia> using BenchmarkTools

julia> foo() = rand((1.3, 1))
foo (generic function with 1 method)

julia> @btime foo()
  18.938 ns (0 allocations: 7 bytes)
1.3

julia> @btime foo()
  19.157 ns (0 allocations: 6 bytes)
1

julia> @btime foo()
  19.238 ns (0 allocations: 7 bytes)
1
2 Likes
2

Maybe it’s some kind of average? @allocated foo() for me always returns either 0 (when returning 1) or 16 (when returning 1.3).
I understand that it allocates due to the uncertain return type, but why only when returning 1.3?

2 Likes
3

I guess it is because foo() = rand((1.3, 1)) is not type stable, foo2() = rand((1.3, 1.0)) is and does not allocate.

Chairmarks.jl actually does show why BenchmarkTools.jl shows 0 allocations:

julia> @b foo()
21.483 ns (0.45 allocs: 7.233 bytes)

It’s rounding to 0.

4 Likes
4

As explained, the allocs are because of type instability - the return value is being boxed, which normally requires an allocation.

My guess is that there is a cache of pre-boxed small integers somewhere, so that case doesn’t allocate.

3 Likes
5

Yes Julia has a cache of boxed small ints -512:512 I believe.

4 Likes
6

Empirically, I can confirm that rand((1.3, n)) for n in [-512, 511] indeed does not allocate when returning n, and does for other Int values of n.

3 Likes
7

Rounding down because one case was preallocated makes sense. This is what happens for non-Ints, which implies rounding of a fractional bytes estimate that I still don’t really get:

julia> foo() = rand((1.3, 1im))
foo (generic function with 2 methods)

julia> @btime foo()
  26.780 ns (1 allocation: 23 bytes)
1.3

julia> @btime foo()
  26.633 ns (1 allocation: 23 bytes)
0 + 1im

julia> @b foo()
64.078 ns (1 allocs: 22.576 bytes)

julia> @b foo()
77.885 ns (1 allocs: 22.513 bytes)

More importantly, does anyone know why there seems to be actually 0 allocations for a similar type-unstable method that doesn’t use rand? Squinting at the @code_llvm suggested to me both methods store 1.3 and 1im as hidden globals.

julia> foo(x, y) = x > y ? 1.3 : 1im
foo (generic function with 2 methods)

julia> @btime foo($3, $5)
  2.600 ns (0 allocations: 0 bytes)
0 + 1im

julia> @b foo(3, 5)
1.178 ns

julia> Base.return_types(foo, typeof.((3, 5)))
1-element Vector{Any}:
 Union{Complex{Int64}, Float64}

julia> Base.return_types(foo, ())
1-element Vector{Any}:
 Union{Complex{Int64}, Float64}