Is there a way to find out free disk space equivalent to psutil.disk_usage('/') in python

General Usage
1

Is there a way to find out free disk space in drive C: both on windows and linux ?
On python I would that by calling
psutil.disk_usage(‘C:’).free

2

It seems not…

If you are OK with using python from Julia you could probably use:

julia> using PyCall

julia> @pyimport shutil

julia> shutil.disk_usage("/")

If you like to play with lower level (see doc) you could write something similar to:

julia> function disk_free_space(path)
          if Sys.islinux()  # maybe isunix would work here too?
            statfs = zeros(UInt64, 11)  # warning: I am not sure all platform use unsigned long for all fields. See `man statvfs`
            ccall((:statvfs, "libm.so.6"), Cint, (Cstring,Ptr{Cvoid}), "/", statfs)
            convert(Int, statfs[1]*statfs[5])
          elseif Sys.iswindows()
             # you could call GetDiskFreeSpaceExA from kernel32.dll 
             # something like ->
             # ccall( (:GetDiskFreeSpaceExA, "kernel32.dll"), stdcall, ... )  
             # sry I have no windows to try
          else throw(ErrorException("Unsupported platform"))
          end
       end
3

Thanks I like the low level solution…

4

So I made the silly mistake of blindly copying this code a few months back and incorporating it into a much larger code set. Sometime later I experienced a segfault that would occur during garbage collection on very rare occasions. This become really annoying when running many instances of said code and cost me a lot of time. The error ultimately was due to statvfs not returning only 11 values as UInt64 but some extra bytes as well. The moral of the story if you use this code pay attention to Liso’s warning:

warning: I am not sure all platform use unsigned long for all fields. See man statvfs

note also that man statvfs at least in my case on Ubuntu 18.04 gives obscure type names of fsblkcnt_t and fsfilcnt_t and even then says it’s only the approximate structure. Personally I don’t really need this call to be efficient so I’m using a df . call now.

1 Like
5

To expand on what Greg said…

The trick to calling this from julia is to find the real definition of statvfs in the system headers (the man page only says the struct is “approximately as follows …”). On my system, it’s in /usr/include/x86_64-linux-gnu/bits/statvfs.h

But even then, there’s several #ifdefs which might or might not be turned on. Running the following C++ program gives you the true size of everything:

#include <iostream>
#include <sys/statvfs.h>

int main()
{
    struct statvfs fs;
#define PRN_SZ(x) std::cout << "sizeof(" #x ") = " << sizeof(x) << "\n";
    PRN_SZ(fs);
    PRN_SZ(fs.f_bsize);    /* Filesystem block size */
    PRN_SZ(fs.f_frsize);   /* Fragment size */
    PRN_SZ(fs.f_blocks);   /* Size of fs in f_frsize units */
    PRN_SZ(fs.f_bfree);    /* Number of free blocks */
    PRN_SZ(fs.f_bavail);   /* Number of free blocks for unprivileged users */
    PRN_SZ(fs.f_files);    /* Number of inodes */
    PRN_SZ(fs.f_ffree);    /* Number of free inodes */
    PRN_SZ(fs.f_favail);   /* Number of free inodes for unprivileged users */
    PRN_SZ(fs.f_fsid);     /* Filesystem ID */
    PRN_SZ(fs.f_flag);     /* Mount flags */
    PRN_SZ(fs.f_namemax);  /* Maximum filename length */
    PRN_SZ(fs.__f_spare);
    //if(statvfs("/", &fs) == 0) {
    // ...
    //}
    return 0;
}

On my system, the sizeof statvfs is 112 bytes which is way larger than 11xUInt64. So that can result in some fun memory corruption when the system tries to write into fs.f_namemax or something (there’s also private padding bytes in __f_spare which could be zeroed for all we know). Basically, the call to statvfs can trample over some random memory. In the crash from Core dump inside LasIO · Issue #30 · visr/LasIO.jl · GitHub that happened to be memory that the GC was using to track another object’s type. When that object eventually goes out of scope the GC tried to collect it and things blew up.

TBH I’m not sure what the “correct” and robust way to deal with system specific structs in headers like these are. The shape and size of statvfs can vary between different architectures.

1 Like
6

If I remember well and if it is still true then Julia is using libuv library. It was problem that libuv missed this function, see for example this issue, but it seems to be resolved in this commit from august 2019.

It probably could be solved by ccall proper function from julia’s libuv. (optimally in julia standard ibrary).

1 Like
7

Yes that’s a much better option. Looks like Julia’s libuv may not be carrying that patch yet though. See

1 Like
8

Filed an issue about this here:
https://github.com/JuliaLang/julia/issues/37038

1 Like