C struct garbage collection not run frequently enough

mattsignorelli
Thu, 11 Jul 2024 15:38:28 +0000
General Usage

I have a Julia interface to a C library where each C struct represents a number. In Julia, the mutable struct just contains a Ptr to the C struct, and the finalizer calls the C destructor.

These C structs can be very memory intensive (they represent Taylor series), but I’ve noticed Julia’s GC does not seem aware of how much memory is actually being used by the C code.

For example, if I call the function:

function foo(x)
  for i = 1:10000
    # operations using the C number type
  end
end

while keeping an eye on top command on my Mac command line, Julia’s memory usage can skyrocket to 5+ GB, and then if I call it again go up to 10+GB, etc until 30+ GB (I’m on a Mac M2 Ultra).

If I instead call the function

function foo(x)
  for i = 1:10000
    # operations using the C number type
    GC.gc(false)
  end
end

there is no skyrocketing of the memory, and in fact, the speed is basically unchanged.

So my questions are

1. Why is this happening? Is Julia’s GC not aware of how much memory is actually being used by the C code?

2. If it is unaware, then how can I tell the GC at object creation in a performant way how much memory is actually being used? If it is aware, then how can I fix this?

29 posts - 7 participants