Is varinfo() designed to not report size of an array of sparse matrices?

General Usage Performance
,
1

Memory usage shoots up after computing two variable/references, D & Rj_mats. For these two, varinfo() reports:

D	6.000 MiB	262144×262144 SparseMatrixCSC{Int64,Int64}
Rj_mats	10.039 KiB	256-element Array{SparseMatrixCSC{Int64,Int64},1}

Curios about memory usage, I saved these two in separate files as below:

@save "blah"*"_Rj_mats"*".bson" Rj_mats
@save "blah"*"_D"*".bson" D

The file sizes reported by OS are:
1.44 GB (1,512,503 KB) for blah_Rj_mats.bson
6.00 MB (6,145 KB) for blah_D.bson

varinfo() was invoked after Rj_mats is initialized inside a function as below and filled up in a subsequent loop (within the same function):

global Rj_mats = Array{SparseMatrixCSC{Int, Int}, 1}(undef,n);

Looking forward to comments, critique and helpful pointers.
Thanks for reading!

2

I think you want Base.summarysize. As stated in ?varinfo

The memory consumption estimate is an approximate lower bound on the size of the internal structure of the object.
1 Like
3

Thanks @dmbates for the pointer. Here is what I get:

Base.summarysize(Rj_mats) returns 10280
Base.summarysize(D) returns 6291624

However, following gives a more informative answer of 1548662784.

sum([Base.summarysize(Rj_mats[i]) for i in 1:length(Rj_mats)])

My versioninfo():

Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Environment:
  JULIA_NUM_THREADS = 3