Seeking clarification on Ptr vs Ref in ccall

General Usage
1

Suppose I have a C code

int sum(int *a, int n){
	int s = 0;
	for(int i=0; i<n; i++){
		s += a[i];
	}
	return s;
}

and I can call this from Julia as

julia> ccall((:sum, "./libsum.so"), Cint, (Ref{Cint},Cint), Cint[1,2,3], 3)
6

julia> ccall((:sum, "./libsum.so"), Cint, (Ptr{Cint},Cint), Cint[1,2,3], 3)
6

which of these two is the recommended form? The section Calling C and Fortran Code · The Julia Language suggests

For C code accepting pointers, Ref{T} should generally be used for the types of input arguments

but I have come across many libraries using Ptr{T} as the argument type, with seemingly no difference in the outcome. My limited (And somewhat naive) understanding of this is that Ref{T} ensures that the GC doesn’t clear the underlying memory, whereas Ptr{T} offers no such guarantee. This would suggest that Ref{T} is the safer option.

Could someone suggest the correct form here? And is there a reason to prefer Ptr over Ref in certain scenarios?

1 Like
2

If the memory is managed by Julia, Ref is preferred.

The actual difference is that Base.cconvert will convert to a Base.RefArray before converting to a pointer.

julia> Base.cconvert(Ref{Cint}, Cint[1,2,3])
Base.RefArray{Int32, Vector{Int32}, Nothing}(Int32[1, 2, 3], 1, nothing)

julia> Base.cconvert(Ptr{Cint}, Cint[1,2,3])
3-element Vector{Int32}:
 1
 2
 3
3

In Initialize buffer for ccall - #2 by stevengj, the suggestion seems to be to use Ptr. Would it be more correct to pass a Ref in this instance?

4

If you have an Array object that you are passing, then Ptr and Ref will have essentially the same effect. If you don’t have an Array already, then Ref{T} allows you to pass a T object and it will create the Ref for you. You need to explicitly create an arg = Ref{T}() object if you want to inspect the value of arg[] after the call, but this is potentially cheaper than allocating an Array.

Semantically, a Ref{T} object acts like a zero-dimensional container of length 1 in Julia, so I think it’s clearer to use it in ccall only if the argument is used as a pointer to a single value, not a whole array. That’s really what Ref was designed for historically in Julia (it came long after Ptr).

So, in your sum function I would tend to pass a as Ptr{Cint} for a pointer to an arbitrary-length array. However, if you also passed a pointer to the result (= 1 number), I would use Ref for that argument. That is, if your C function looked like:

void sum(int *result, const int *array, int n) {
    /* ... store sum of array[i] values in *result ... */
}

then I would call it as:

a = Cint[1,2,3]
result = Ref{Cint}()
@ccall libsum.sum(result::Ref{Cint}, a::Ptr{Cint}, length(a)::Cint)::Cvoid
sum = result[]

Semantically, a RefArray acts like a 0-dimensional view of a single element, and can be used to reference an arbitrary element of an array:

julia> a = [3,1,4]
3-element Vector{Int64}:
 3
 1
 4

julia> r = Ref(a, 3)
Base.RefArray{Int64, Vector{Int64}, Nothing}([3, 1, 4], 3, nothing)

julia> r[]
4

julia> length(r)
1

julia> ndims(r)
0

and I would tend to use it in that context (for passing pointers to individual elements, not the whole array).

5

Thanks, that’s helpful. Does that mean the documentation stating

For C code accepting pointers, Ref{T} should generally be used for the types of input arguments

may clarify that this is unnecessary for julia arrays, and a Ptr may be passed instead? Tbh the documentation is quite vague on this, and a few examples suggesting the canonical use cases would be quite helpful.

6

Yes, I think that should be clarified.

7

Sorry for the slight high-jacking, but also in section Calling C and Fortran Code · The Julia Language this first was cryptic for me:

When calling Fortran, all inputs must be passed by pointers to heap- or stack-allocated values, so all type correspondences above should contain an additional Ptr{…}orRef{…} wrapper around their type specification.

After trial and error my conclusion was that in Fortran all arguments, even scalar ones, need to be passed as Ptr{..} or Ref{..}, same as sum in the C example above, and even so when the argument is intent(in). They also need to be initialized as Ref(...). Example:

a = Cint[1,2,3]
result = Ref(Cint(0))
len = Ref(Cint(length(a)))
@ccall libfortransum.sum(result::Ref{Cint}, a::Ptr{Cint}, len::Ref{Cint})::Cvoid
sum = result

Especially for Fortran I think that perhaps it should be clarified that a “wrapper” in the actual ccall is not sufficient. All scalars, whether input or output, must be initialized as Refs, and there Ptr does not work.

8

After Base.cconvert, Base.unsafe_convert would then convert the Base.RefArray to a Ptr:

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-beta3 (2023-01-18)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> Base.cconvert(Ref{Cint}, Cint[1,2,3])
Base.RefArray{Int32, Vector{Int32}, Nothing}(Int32[1, 2, 3], 1, nothing)

julia> Base.unsafe_convert(Ref{Cint}, Base.cconvert(Ref{Cint}, Cint[1,2,3]))
Ptr{Int32} @0x00007fbb60190860

julia> Base.cconvert(Ref{Cint}, Cint[1,2,3])
Base.RefArray{Int32, Vector{Int32}, Nothing}(Int32[1, 2, 3], 1, nothing)

julia> @which Base.cconvert(Ref{Cint}, Cint[1,2,3])
cconvert(T::Type, x)
     @ Base essentials.jl:492

julia> @which Base.convert(Ref{Cint}, Cint[1,2,3])
convert(::Type{Ref{T}}, x::AbstractArray{T}) where T
     @ Base refpointer.jl:117

julia> @which Base.unsafe_convert(Ref{Cint}, Base.cconvert(Ref{Cint}, Cint[1,2,3]))
unsafe_convert(::Type{Ref{T}}, x::Ref{T}) where T
     @ Base refpointer.jl:101

For Array, perhaps it does not matter. Might it matter for other kinds of AbstractArray?

9

No, I don’t think so.

In any case where you need a C-compatible pointer to multiple elements, not just one, then allocatedinline(T) must be true and it just calls pointer(array, i), which in turn just calls unsafe_convert(Ptr{T}, array). So in the end it should be equivalent to Ptr{T}.