Objects not being finalized until exit() when using multithreading. Is this a bug?

General Usage
,
1

Here’s a script which defines a mutable struct Foo, creates 10 of them, loops over them using Threads.@threads and then calls GC:

mutable struct Foo
    value::Int
    function Foo(value)
        finalizer(new(value)) do x
            println("finalizing ", x)
        end
    end
end

function test()
    objs = map(Foo, 1:10)
    Threads.@threads for i = 1:10
        identity(objs[i])
    end
    nothing
end


println("test()")
test()
println("GC.gc()")
GC.gc()
println("GC.gc()")
GC.gc()
println("GC.gc()")
GC.gc()
println("GC.gc()")
GC.gc()
println("GC.gc()")
GC.gc()
println("exit()")
exit()

Here’s the output when using 2 threads (julia -t2 script.jl):

test()
GC.gc()
GC.gc()
GC.gc()
GC.gc()
GC.gc()
exit()
finalizing Foo(10)
finalizing Foo(9)
finalizing Foo(8)
finalizing Foo(7)
finalizing Foo(6)
finalizing Foo(5)
finalizing Foo(4)
finalizing Foo(3)
finalizing Foo(2)
finalizing Foo(1)

What is surprising to me is that all the Foo objects are not finalized until the script exits. AFAICT these objects should be unreachable after test() completes, so these should be finalized after a call to GC.gc().

If I change test() to any of these variants

# loop is single-threaded
function test()
    objs = map(Foo, 1:10)
    for i = 1:10
        identity(objs[i])
    end
    nothing
end

# no loop at all
function test()
    objs = map(Foo, 1:10)
    # Threads.@threads for i = 1:10
    #     identity(objs[i])
    # end
    nothing
end

# loop does nothing
function test()
    objs = map(Foo, 1:10)
    Threads.@threads for i = 1:10
        # identity(objs[i])
    end
    nothing
end

then as expected the Foos are finalized after calling GC.gc():

test()
GC.gc()
finalizing Foo(10)
finalizing Foo(9)
finalizing Foo(8)
finalizing Foo(7)
finalizing Foo(6)
finalizing Foo(5)
finalizing Foo(4)
finalizing Foo(3)
finalizing Foo(2)
finalizing Foo(1)
GC.gc()
GC.gc()
GC.gc()
GC.gc()
exit()

So my question is - is this a bug or am I going mad? I have read the caveats around multi-threading but it’s not clear to me if any of those apply here.

It seems to me that whatever Threads.@threads expands to is somehow capturing the objs array for the whole lifetime of the program, so is only finalized when Julia exits. I took a look at the output of @macroexpand1 Threads.@threads ... and it does indeed create a function which captures objs, but it’s all within the scope of the test() function so unclear why objs is still reachable after test() finishes.

2

I think this is a known problem, an old issue: Thread-local current_task keeps garbage alive · Issue #40626 · JuliaLang/julia · GitHub

1 Like
3

Aha, thank you!