Example of registering a finalizer for unsafe_wrapped data from ccall

New to Julia
,
1

Hi, I’m using ccall on a C library that returns a pointer to some data that was malloced in the library and requires a separate API call to close the resource that the data came from. The dataset could be quite large (a couple of GB) so I’d rather use unsafe_wrap in julia, but I’m not familiar with how to implement finalizers. The julia docs imply that a finalizer can only be registered for a mutable struct. Here’s a fictional gist of my application:

function getCdata(recordNum::Integer)
    handle = @ccall mylib.openRecord(recordNum::Cint)::Cint
    if handle < 0
        error("failed to open record ", recordNum)
    end
    leng = @ccall mylib.getRecordSize(handle::Cint)::Cint
    dataPtr = @ccall mylib.getRecordData(handle::Cint)::Ptr{UInt8}
    dataRef = unsafe_wrap(Array, dataPtr, leng)
    closefunc() = @ccall mylib.closeRecord(handle::Cint)::Cvoid
    return (dataRef,closefunc)
end

So the function opens a record via the c library and wraps the array in dataRef. It also returns a closure that can call the api close function, but how do I arrange that the garbage collector will call this (or do something to the same effect) when I’m done using dataRef?

2

Should be

return finalizer(dataRef) do ref
   closefunc()
end
3

Wow, thanks, that was simpler than expected. So do you know if I need the closure function at all or can I use the do block for this directly, i.e.

return finalizer(dataRef) do
    @ccall mylib.closeRecord(handle::Cint)::Cvoid
end

or would the handle go out of context?

4

I think that’s fine.

5

OK thanks again. Well, it looks like the finalizer was registered but threw an error on julia exit. Not sure if it’s me or a julia bug.

error in running finalizer: MethodError(f=Main.var"#6#7"{Int32}(handle=5), args=(Array{UInt8, 1}(dims=(191554446,), mem=Memory{UInt8}(191554446, 0x7f377f318010)[0x89, 0x48, 0x44, 0x46, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, … (screenfuls of numbers ad nauseum, probably dumping the entire memory block)

6

I think the passed closure requires one argument (see the MWE in the docstring of finalizer; was my bad, I updated the above answer). Try

return finalizer(dataRef) do ref
    @ccall mylib.closeRecord(handle::Cint)::Cvoid
end
7

Thanks, that worked, albeit surprisingly. I think this relates back to my original question: the help text indicates that finalizers can only be registered for mutable struct types. This seems quite restrictive. In the above example dataRef is just an array, so I’m surprised I “got away with it”.
I don’t find the finalizer docstring examples really that helpful because they only show the use of a finalizer to produce a debug statement, not how it could be used in practice for really “finalizing” an object

8

Arrays qualify, although they are more magical than your typical mutable struct, in particular before Julia 1.11.

julia> ismutabletype(Array)
true

julia> isstructtype(Array)
true
9

Ah, that explains it! I didn’t expect that.
thanks!

10

In Julia 1.11 I think this should be:

return finalizer(dataRef.ref.mem) do x
   closefunc()
end

Because the wrapping Array can get finalized while the underlying memory is still being used.
This is what Mmap.jl does.

Another tricky thing is that as far as I can tell, finalizers can be concurrently run on different threads, so you need to ensure the C library is thread-safe for this to work. PythonCall.jl has some example finalizers for dealing with non-thread-safe C libraries.

1 Like