A clever way to find the available memory the host currently has free/available?

General Usage
, ,
1

Writing a program which uses single files to large to fit in the memory of normal desktops or laptops, I thought it would be nice to dispatch on the available memory relative to some expected required memory. In case the available memory is not enough to fit the data and overhead, the function should dispatch and use load only the first couple of entries. This preview should be enough to write a analysis routine on a laptop. Once everything is figured out, the routine could be submitted to a HPC cluster.

I have read the docs but need some clarification.
Some thing similar to vmstat -s | grep "free memory" | awk '{print $1 $2}'
I came up this

function getFreeMemory()
    run(pipeline(`vmstat -s`, `grep "free memory"`, `awk '{print $1}'`))
end

However, running: freeMem = getFreeMemory() prints the number and returns the ProcessChain:

8064912
Base.ProcessChain(Base.Process[Process(vmstat -s, ProcessExited(0)), Process(grep 'free memory', ProcessExited(0)), Process(awk '{print $1}', ProcessExited(0))], Base.DevNull(), Base.DevNull(), Base.DevNull())

How could I catch the number (and discard the ProcessChain)?

2

https://docs.julialang.org/en/v1/manual/running-external-programs/#Running-External-Programs-1
therefore you may use

function getFreeMemory()
    read(pipeline(`vmstat -s`, `grep "free memory"`, `awk '{print $1}'`),String)
end

Didn’t checked it, because I don’t have a linux shell currently. But I checked with a Cygwin bash this and it worked:

julia> function getFreeMemory()
           read(pipeline(`ls`, `grep "Dok"`),String)
       end
getFreeMemory (generic function with 1 method)

julia> getFreeMemory()
"Dokumente\n"
3

Are you aware of the Base.Sys.free_memory function? If I’m understanding what you’re trying to do that might be able to give you the information you need without needing to run a pipeline of external programs.

2 Likes
4

Thanks. I did not know about it.
Here a previous discussion, and the docs.

5

This works as well.
For completeness:

parse(Int, read(pipeline(`vmstat -s`, `grep "free memory"`, `awk '{print $1}'`),String)[1:end-1])

I’ll stay with the Base.Sys.free_memory solution.