Weird behaviour of GC with multithreaded array-access

wilhelm
Fri, 31 Jan 2025 16:37:34 +0000
General Usage

Hi!

I recently worked with some big arrays and used multithreading to access and fill these arrays, but after the calculation it seemed like the GC did not manage to clear the ram used by the “non-main” thread.

MWE:

function RAMHungry(NumIter::Int64)
    BigVector::Vector{Float64} =  zeros(NumIter)
    Threads.@threads for i in eachindex(BigVector)
        BigVector[i] = 1.0
    end
    return nothing
end

# N = 1000000000

RAMHungry(N)

where N should be set to a value big enough to see a considerable RAM-use in your system monitor.

After executing the function the memory is not automatically freed, but even after manually running

GC.gc()

the memory is still not freed.

After some trial and error I did find a solution:

Threads.@threads for i in 1:Threads.nthreads()
    GC.gc()
end

I found this quite confusing, as Julia will even crash if further memory is allocated.
As such it seems like, to me, that the main thread does not have control of “system-wide” (all threads) Garbage Collection.

Is this intended behavior, and is the solution above the intended way of solving the issue?

( I have run these tests using Julia Version 1.11.3 (2025-01-21) in VSCode, and I am on Linux Mint )

EDIT:
I tried the MWE in terminal and thought the problem only applied to VSCode, I have now tested again, and the problem persisted also in terminal

If I have missed something, please tell me!

3 posts - 3 participants