Is there a package to list memory consumption of selected data objects?

General Usage
16

I couldn’t find out how to measure dictionary memory, and I wasn’t comfortable with large heterogeneous tuples that can store its elements in various ways, so I went with a simpler Vector{Any} as the container.

It appears that Base.summarysize doesn’t actually report all the memory used by a Vector{Any}. I know there should be boxes containing element type information, but it only seems to count the elements and the vector’s pointers to them. Also, if you allocate a v = collect(1:100000) then empty!(v), the underlying buffer does not shrink if I recall correctly, but sizeof and summarysize reports a reduction to minimal memory. So summarysize might actually be unsuitable for measuring allocated heap memory; maybe we could say it measures the portion of allocated memory that represents accessible data?

Still, this might make it easier to remove the overhead of the Vector{Any} containing the @locals instances. Bear in mind the following only worked on a few small examples, I have not rigorously tested this and do not know how.

julia> function valuessize(v::Vector{Any})
           (Base.summarysize(v) # doesn't seem to count boxes
           - Base.summarysize(Any[]) # Vector overhead
           - sizeof(Int)*length(v) ) # element pointers
       end
valuessize (generic function with 1 method)

julia> function valuessize(d::Dict{Symbol, Any})
           valuessize(collect(values(d)))
       end
valuessize (generic function with 2 methods)

julia> x = Dict{Symbol, Any}(:x => 1, :y => 2.3, :z => [3])
Dict{Symbol, Any} with 3 entries:
  :y => 2.3
  :z => [3]
  :x => 1

julia> valuessize(x)
64

julia> sum(Base.summarysize.(values(x))) # elements don't share data
64

julia> Base.summarysize(collect(values(x))) # plus Vector{Any} overhead
128

julia> Base.summarysize(values(x)) # plus values/Dict{Symbol,Any} overhead
528
1 Like