Reference counting and disposables

General Usage
,
1

In ResourcePools.jl/pooled_abstract_array.jl at feature/multi-threading · IHPSystems/ResourcePools.jl · GitHub, I have two concepts - in addition to pooled resources / pools of resources:

  • Reference counted instances - that can be retain!'ed and release!'ed - and where something should happen when the reference count reaches zero. Thread-safe.
  • Disposables - instances that react to dispose!, e.g., by cleaning up memory allocated outside Julia.

I have yet to find these quite common concepts in Base. Are they there somewhere or should they be split out into separate tiny packages?

2

Another hit for looking up .NET-like disposables (IDisposable): Iterators and resource management · Issue #22466 · JuliaLang/julia · GitHub

3

Related to Disposables: LLVM.jl defines a @dispose macro “for disposing resources without do-block syntax”:

Helper macro for disposing resources (by calling the LLVM.dispose function for every
resource in reverse order) after executing a block of code. This is often equivalent to
calling the recourse constructor with do-block syntax, but without using (potentially
costly) closures.

Introduced in Introduce at-dispose to replace do-block constructors. by maleadt · Pull Request #309 · maleadt/LLVM.jl · GitHub

4

See also this (concurrent) discussion: https://julialang.zulipchat.com/#narrow/stream/137791-general/topic/Customizable.20or.20static.20memory.20allocation/near/383702361

1 Like
5

@maleadt Care to comment on the prospect of “Disposables”? E.g. something outside LLVM that would do sort of the same thing…

I had something like the following (slightly bad idea) on my mind until looking at LLVM:

module Disposables

export dispose!, with_disposal

function dispose! end # dispose!(::T)

function with_disposal(f::Function, args...; kwargs...)
    try
        f(args...; kwargs...)    
    finally
        for arg in args
            dispose!(arg)
        end
    end
end

# test suite in a non-exported Testing sub-module for checking implementations of informal interface

# implementations of dispose!(::T) for Base types like arrays etc. ... - in a package extension or in a separate "DisposableBase" package

end # module

Issues:

  • dispose! needs to consider interaction with GC - the same resource should not be disposed of twice in case someone hooks up dispose! to GC finalization with finalizer - similar to how IDisposable works in .NET (cf. IDisposable.Dispose(true) vs IDisposable.Dispose(false))
  • dispose! must be implemented in a thread-safe manner - without risking dead locks. E.g. in relation to thread-safe reference counting (with both “disposable” and “reference countable” having a lock…)
  • with_disposal seems problematic: Should all arg in args be disposed of? (probably not) Should they be disposed of in-order?

Something like the @dispose macro is probably better than with_disposal…