Is map(f,collection) memory inefficient?

Internals & Design
1

Hi All,

I used a 1MB array and applied map on it for some computation. It shows me an estimated allocation of 8MB. Can this happen or it’s some issue in reporting of BenchmarkTools package?

From pure intuition one can say the allocation should be the size of the input array. 8 times is a bit unlikely.

julia> @benchmark map(x -> 2x, arr)
BenchmarkTools.Trial: 
  memory estimate:  8.00 MiB
  allocs estimate:  3
  --------------
  minimum time:     1.992 ms (0.00% GC)
  median time:      2.309 ms (0.00% GC)
  mean time:        2.592 ms (7.45% GC)
  maximum time:     54.632 ms (93.92% GC)
  --------------
  samples:          1923
  evals/sample:     1

regards,

Sambit

2

Since arr is a global variable at the REPL, its type could change at any time. So using it in a benchmark will give misleading results because Julia also has to deal with the potential type-instability. You need to splice in the current value with $ like this:

@benchmark(x -> 2x, $arr)

Does that change your allocation results at all?

3

No change

julia> @benchmark map(x -> 2x, $arr)
BenchmarkTools.Trial: 
  memory estimate:  8.00 MiB
  allocs estimate:  3
  --------------
  minimum time:     2.166 ms (0.00% GC)
  median time:      2.438 ms (0.00% GC)
  mean time:        2.970 ms (9.40% GC)
  maximum time:     57.251 ms (93.92% GC)
  --------------
  samples:          1679
  evals/sample:     1
4

Please ignore. Didn’t realize the 2x will be a UInt and not UInt8.

julia> @benchmark map(x -> UInt8(2x), $arr)
BenchmarkTools.Trial: 
  memory estimate:  1.00 MiB
  allocs estimate:  3
  --------------
  minimum time:     936.873 μs (0.00% GC)
  median time:      951.316 μs (0.00% GC)
  mean time:        978.969 μs (1.52% GC)
  maximum time:     1.825 ms (0.00% GC)
  --------------
  samples:          5101
  evals/sample:     1

This fixes the issue.

1 Like