In the docs explaining the method of “unsafe_wrap(Array, pointer::Ptr{T}, dims; own = false)”, It says that “… own optionally specifies whether Julia should take ownership of the memory, calling free on the pointer when the array is no longer referenced…” But I didn’t find any relevant method to conduct the “free”, Any help?
The answer depends on where you got the pointer from. If you allocated with Libc.malloc then you should use Libc.free.
In general, for objects that Julia “owns”, you should let the garbage-collection system decide when to deallocate the memory. However, some closely related functions for freeing up resources are finalize (which runs finalizers immediately on a particular object) and GC.gc (to run the whole garbage collector immediately).
Let me take an example to clarify my question, a piece of code like this:
a = [1, 2]
p_a = pointer(a)
t = unsafe_wrap(Array{Int64}, p_a, (2, 1), own = true)
# It results in t = [1, 2]
finalize(p_a)
a[2] = 5
# It results in : t = [1, 5]
I intent to keep t = [1, 2] after a[2] = 5.
If you use own=true, I don’t think you have to (or even can) specify free.
Otherwise, (own=false), then then there should be a pair of alloc and free functions that you use. And you use the free that corresponds to the alloc that you used to get the array. The latter is what the response above is referring to.
EDIT: I wrote this without seeing your most recent post above. I made another comment in response to it below.
It might help if you explain the problem in more general terms. Forgetting about pointers and ownership for the moment, what are you trying to accomplish?
Regarding the last bit of code, you are not allocating a new buffer and copying the contents from the original. You only have one buffer (or maybe none, if the memory is freed). And you changed an element in this single buffer.
1 Like
My application is mainly relevant to calling for Julia-creating C shared library in an asynchronous parallel manner. Specifically, if some data passed to a Julia function was changed before the called function returned, then the proceeding Julia function will behave unexpectedly. I thought that the keyword of “own = true” in unsafe_wrap method can safely own the data without explicitly “deepcopy” the data.
Do not free the pointer p_a. Julia’s garbage collection will do this when a can no longer be accessed. If you free it, there will be a double free situation and Julia will likey crash.
Just do t = copy(a). I do think there is any way around this. You may need to use GC.@preserve to make sure Julia does not free the memory when you pass it to C until it returns.
Alternatively, alloc the memory using Libc.malloc and wrap that. You can then use unsafe_wrap to treat the allocated memory like an array. The memory will not be freed until you use Libc.free.
p_a = Ptr{Int}(Libc.malloc(sizeof(Int)*2))
a = unsafe_wrap(Array, p_a, (2,1); own=false)
a .= (1,2)
# pass a or p_a to C
# do something else
# get the results back from C sometime later
a = nothing
Libc.free(p_a)
2 Likes
Thank you very much for the detailed answer! I can understand that your answer is mainly discussing two ways of safely passing data to the “ccall”, and also gives a hint that data copying seems to be unavoidable if data passed to a Julia shared library is desired to be safely used.
I’ve learned a lot from your answer!
own = true is wrong here because this is not a pointer to memory that you allocated with malloc, so calling free on p_a is wrong and is likely to lead to crashes.
finalize(p_a) doesn’t do anything because a pointer is essentially just a number and doesn’t have a finalizer.
And a and t refer to the same memory, so modifying one affects the other.
4 Likes
The following crashes julia reliably on my machine:
function badwrap()
a = [1, 2]
p_a = pointer(a)
t = unsafe_wrap(Array{Int64}, p_a, (2, 1), own = true)
nothing
end
for _ in 1:10^5
badwrap()
end
It should be clear now why this crashes.
The line a = [1, 2] does two things. 1) It allocates storage (and writes values) and 2) binds the variable a to this storage (ie memory).
Suppose we didn’t call unsafe_wrap. Each time control exits badwrap, the variable a goes out of scope. Now there is no way the programmer can access the storage that it was bound to. So the storage is marked for eventual garbage collection.
But, we did call unsafe_wrap, so both a and t are bound to the same storage (maybe with different metadata, but the raw array is the same). Now when control exits badwrap both a and t go out of scope. If they had been created using “normal” Julia constructs, Julia would realize that they both point to the same storage, and so only free that storage once.
But pointer(a) is just a number; eg you can convert it to an Int:
julia> convert(Int, pointer([1,2,3]))
140424361882976
So Julia has no way of knowing where the memory pointed to came from. In particular, it doesn’t know that p_a points to memory already managed by GC. By calling unsafe_wrap with own=true, you have told Julia that GC should manage this memory, that the pointer points to two Ints-worth of storage that the Julia gc may free when t goes out of scope. This functionality was intended for memory that you allocated elsewhere, memory for which you don’t want to call the corresponding free.
The result is that in bad_wrap Julia knows it has to manage the storage that a is bound to and storage that t is bound to. But, it doesn’t know that they point to the same buffer. So it will try to free the same memory twice, which results in a crash.
3 Likes