Relationship between WeakRef and finalizer (building a cache)

General Usage
,
1

I am hoping someone can tell me what I am doing wrong.

Long story short, I am working on a program that does some caching in Julia. My goal is to allow my “cache” structure to be garbage collected as necessary, but any data that is saved in the cache needs to post-processed/written out to disk (I am currently doing this in a finalizer). If there is no cache for a particular object, I pull in the data from disk into a new cache.

I am using a WeakRef to point to my cache. When I need to access the cache and WeakRef is nothing, I just make a new cache. Unfortunately, it seems like my WeakRef gets set to nothing before my finalizer runs. This results in me reading old values from disk (because my cache exists in memory and has not been written to disk in the finalizer). But I can’t access this in-memory cache because the last reference via the WeakRef is now gone.

MWE below.

My expectation was that my ref_to_cache won’t be set to nothing until the struct has been fully finalized…but this does not appear to be the case.

Is there any way to satisfy the following:

  1. Allow my structure to be garbage collected at will
  2. Ensure that a reference to my structure exists unless the finalizer has completed?

This, of course, works fine without threading. Wondering if this is a problem with deferring the finalizer when the lock is not available…or maybe I just don’t understand the relationship between WeakRef and the finalizer.

MWE:

mutable struct Foo
    ref_to_cache::WeakRef
    cache_finalized::Bool # Flag to indicate the the cache finalizer finished
    lock::Base.AbstractLock
end

mutable struct CacheStruct
    foo::Foo
    function CacheStruct(foo::Foo)
        weak_ref_struct = new(foo)
        finalizer(save_cache_finalizer, weak_ref_struct)
        return weak_ref_struct
    end
end

function save_cache_finalizer(ref_struct)
    # Attempt to get the lock, otherwise defer
    # See https://docs.julialang.org/en/v1/manual/multi-threading/#Safe-use-of-Finalizers
    if islocked(ref_struct.foo.lock) || !trylock(ref_struct.foo.lock)
        finalizer(save_cache_finalizer, ref_struct) # Reschedule/delay finalization
        return nothing
    end
    ref_struct.foo.cache_finalized = true
    unlock(ref_struct.foo.lock)
end

foo = Foo(WeakRef(Nothing), true, Base.ReentrantLock())
foo.ref_to_cache = WeakRef(CacheStruct(foo)) # New cache structure
foo.cache_finalized = false

for i = 1:1000
    lock(foo.lock) do
        @assert (foo.ref_to_cache == nothing && foo.cache_finalized == true) || (foo.ref_to_cache != nothing && foo.cache_finalized == false)
    end
    GC.gc()
end