How to get current Julia process memory usage?

General Usage
1

I was debugging a ccall memory leak problem and found the memory usage value in top useful:

image
image1140×46 4.27 KB

Is there a direct method to access this value from Julia? I used Sys.free_memory as a proxy but it suffers from noise introduced by other processes.

2 Likes
2

Made a workaround:

out = read(`top -bn1 -p $(getpid())`, String)
split(split(out,  "\n")[end-1])[6]
1 Like
3

In the same vein, you could also use ps:

julia> run(`ps -p $(getpid()) -o pid,comm,vsize,rss,size`);
4 Likes
4

Since you are using linux you might want to try reading "/proc/$(getpid())/mstat". You can a full description of the fields from:
https://man7.org/linux/man-pages/man5/proc.5.html

From the page:

Provides information about memory usage, measured in pages.
The columns are:

    size       (1) total program size
               (same as VmSize in /proc/[pid]/status)
    resident   (2) resident set size
               (same as VmRSS in /proc/[pid]/status)
    shared     (3) number of resident shared pages (i.e., backed by a file)
               (same as RssFile+RssShmem in /proc/[pid]/status)
    text       (4) text (code)
    lib        (5) library (unused since Linux 2.6; always 0)
    data       (6) data + stack
    dt         (7) dirty pages (unused since Linux 2.6; always 0)
3 Likes
5

pixel27 gives a solution without command:

open("/proc/$(getpid())/statm") do io
    split(read(io, String))[1]
end

But I still wonder if there is a cross platform solution so that memory leak checks can be written in the unit test / CI?

6

I believe it’s hard to determine programatically if there is a memory leak unless you are counting allocation and frees. Many of the allocation libraries allocate chunks of RAM from the OS then feed that out to the program in smaller chunks to improve allocation speed. So while the program is only actively using X amount of RAM Y amount (which is greater than X) will be allocated and I don’t think we as the end developer have much control over releasing that memory back to the system.

1 Like
7

Here’s at least a windows solution:

function memory()
    s = String(read(`tasklist /FI "PID eq $(getpid())" /NH /FO csv`))
    s = replace(s, "\"" => "")
    @info s
    m = strip(split(s, ",")[end])
    m = replace(m, "K" => "1000", "M" => "1000000", "G" => "1000000000")
    parse.(Float64, split(m)) |> prod |> Int
end

(I’m not sure, but it seems to me that Windows counts K as 1000 and not as 1024, etc.)

or this shorter one:

function memory()
    s = String(read(`tasklist /FI "PID eq $(getpid())" /NH /FO csv`))
    m = replace(match(r"""([^"]+)"[^"]+$""", s).captures[1],
        "K" => "1000", "M" => "1000000", "G" => "1000000000")
    parse.(Float64, split(m)) |> prod |> Int
end
3 Likes
8

Just was in need of tracking a memory leak and found out that ‘,’ are not decimal separators and should be discarded. Furthermore, it seems that only “K” appears as dimension. So my new version looks like this:

function memory()
    s = String(read(`tasklist /FI "PID eq $(getpid())" /NH /FO csv`))
    m = replace(match(r"""([^"]+)"[^"]+$""", s).captures[1],
        "K" => "1024", "M" => "1024 1024", "G" => "1024 1024 1024", r"\.|," => "")
    parse.(Int, split(m)) |> prod
end

EDIT: replace both, ‘,’ and ‘.’
EDIT 2: replaced string by regex string

1 Like
9

It does not work when I cut and past it on my REPL, Julia v1.8.4.

julia> function memory()
           s = String(read(`tasklist /FI "PID eq $(getpid())" /NH /FO csv`))
           m = replace(match(r"""([^"]+)"[^"]+$""", s).captures[1],
               "K" => "1024", "M" => "1024 1024", "G" => "1024 1024 1024", "\.|," => "")
ERROR: syntax: invalid escape sequence
Stacktrace:
 [1] top-level scope
   @ none:1

julia>     parse.(Int, split(m)) |> prod
ERROR: UndefVarError: m not defined
Stacktrace:
 [1] top-level scope
   @ REPL[2]:1

julia> end
10

I forgot to add a regex string, sorry. I edited the post above which does work now.

Meanwhile I found that the tasklist command seems to always return answers in true KB.
In that case, I’d propose to have memory always return the result in KB.

function memory()
    s = String(read(`tasklist /FI "PID eq $(getpid())" /NH /FO csv`))
    parse(Int, replace(match(r"""([^"]+)"[^"]+$""", s).captures[1], r"[.,K]" => ""))
end
11

Just needed this on a Docker where no ps is installed, so that information came in handy.
I build a small function which also captures the pagesize and returns the memory size in Bytes or one of “KB”, “MB”, “GB”.

function memory_usage(unit::String = "Bytes")::Union{Float64, Int}
    exponents = Dict("Bytes" => 0, "Byte" => 0, "B" => 0, "KB" => 1, "MB" => 2,  "GB" => 3)

    exponent = exponents[unit]
    pagesize = parse(Int, read(`getconf PAGESIZE`, String))
    meminfo = parse(Int, split(read(`cat /proc/$(getpid())/statm`, String))[1])
    memsize = exponent == 0 ? meminfo * pagesize : meminfo * pagesize / 1024^exponent

    return memsize
end

# example usage

memory_usage()
memory_usage("GB")