Weird behaviour of GC with multithreaded array-access

General Usage
,
1

Hi!

I recently worked with some big arrays and used multithreading to access and fill these arrays, but after the calculation it seemed like the GC did not manage to clear the ram used by the “non-main” thread.

MWE:

function RAMHungry(NumIter::Int64)
    BigVector::Vector{Float64} =  zeros(NumIter)
    Threads.@threads for i in eachindex(BigVector)
        BigVector[i] = 1.0
    end
    return nothing
end

# N = 1000000000

RAMHungry(N)

where N should be set to a value big enough to see a considerable RAM-use in your system monitor.

After executing the function the memory is not automatically freed, but even after manually running

GC.gc()

the memory is still not freed.

After some trial and error I did find a solution:

Threads.@threads for i in 1:Threads.nthreads()
    GC.gc()
end

I found this quite confusing, as Julia will even crash if further memory is allocated.
As such it seems like, to me, that the main thread does not have control of “system-wide” (all threads) Garbage Collection.

Is this intended behavior, and is the solution above the intended way of solving the issue?

( I have run these tests using Julia Version 1.11.3 (2025-01-21) in VSCode, and I am on Linux Mint )

EDIT:
I tried the MWE in terminal and thought the problem only applied to VSCode, I have now tested again, and the problem persisted also in terminal

If I have missed something, please tell me!

2 Likes
2

Can reproduce this, aftter running

 RAMHungry(500000000)

a few times, The Julia memory usage is consistently at 8GB. Calling GC.gc() multiple times only reduce it to 4.5GB, and calling GC.gc() in a thread-loop reduce it to 450MB

3 Likes
3

This is an annoying quirk of the current threading system, where the threads hold on to their task until they find something else to run. This causes Tasks to have an annoyingly long lifetime if you don’t spawn many of them.

We have some ideas on how to fix it, but it’s not super duper easy.

7 Likes