Give hints to Julia's GC about size of memory allocated through C calls

ccall

#1

In SCIP.jl, we build a wrapper for a C library. There’s a Julia type SCIPModel that basically holds a Pointer to a C struct. In the constructor, we call a corresponding initializer of the C library that allocates significant memory. The destructor is then called in the finalizer of our Julia type.

Generally, the garbage collection works find: At some point, when our Julia type is out of scope, it will be finalized and the memory of the C library freed as well. But I’ve noticed that this garbage collection is sometimes delayed a lot, to the point that my machine freezes because it runs out of memory (see PR #34.)

My guess is that Julia knows when to call gc() before it runs out of memory, but it only knows about its own memory, not the implicitely allocated one. So my question is: Can I make Julia’s GC aware of the (rought amount of) memory that belongs to some of its objects?


#2

For CHOLMOD in Julia base, it is using a configuration option in CHOLMOD so that the allocator used is the same one as julia does: https://github.com/JuliaLang/julia/blob/bdef3b57d34ec90781d30f62adf73d048e4a3090/base/sparse/cholmod.jl#L157.

Perhaps you could do something like that if the C library allows for it.


#3

Thanks @kristoffer.carlsson, that is a nice solution. Our C library implements its own (block) memory management and AFAIK the user can not tell it to use another malloc function.