Ccall with pointer argument to array of custom type via an NTuple of immutables

General Usage
1

tl;dr: How to best create a pointer to an array of user-defined type of possibly mutating data when calling into a legacy C API.

I’m trying to Ccall a function of a legacy application in which I’ve embedded Julia to replace the original scripting language. The application wraps values into its own symbol type, a C struct. The C API functions expect an array to those symbols to hold the actual arguments. The contained values might be changed; the array is not changed in terms of memory location and size.

So, I have a value wrapper like the following plus a bunch of constructors

immutable MyType
	data::UInt64      # a union of the value or a pointer to the value
	datatype::Int64   # an enum of the contained type
end

The constructors from the relevant Julia types into that wrapped type work fine.

Then I have a function like the following that wraps any number of Julia values into a NTuple and performs the ccall:

function apicall(fn::Symbol, args...)
    # Convert the arguments into a tuple
    wrapped_args = map(MyType, args)
    nargs = length(args)
    fnname = string(fn)
	
    rarg = Ref{MyType}  # <---- (A)
    # Alternative to the above via a zero-dimensional array to zero-initialize
    #rarg = Array{MyType}(); rarg[] = MyType()

    success = ccall(:apifn, Int64, 
        (CString, Int64, Ptr{Void}, Ptr{Void}),  # <---- (B)
        fnname, nargs, 
        pointer_from_objref(wrapped_args), pointer_from_objref(rarg)) # <---- (C)

    success != 0 && error("Call failed.")
    get(rarg) # Unwrap and return result
end

The above kind of works, but it seems to be prone to errors.

For clarification: The C API implements a signature of

# Variant 1
int apifn(const char*, int narg, MyType args[], MyType* rarg)
# Variant 2
int apifn(int narg, MyType args[], MyType* rarg)

with rarg being the actual result; also wrapped.

The issue I’m having is with lines (A), (B) and (C):
(A): How to best create a reference to MyType which should also be default initialized? Here Ref appears to create unintialized memory.
(B,C): How can I get to the correctly GC managed Ref{MyType} or similar when having such an NTuple of MyTypes?

Thus far, I figured I need to write a Base.cconvert and Base.unsafe_convert method, like it is e.g. done in StaticArrays. But I fail at figuring out how to do that for an NTuple.
Or, is there a more canonical way to achieve something similar?
Many thanks in advance!

2

This is wrong, you at least need Ref{MyType}()

Cstring

Never call pointer_from_objref period.

Use ccall(..., (..., Ptr{Void}, Ref{MyType}, ...), ..., Ref(wrapped_args), rarg, ...) instead.

Ref(MyType())

You don’t.

You shouldn’t define it for NTuple

To make the syntax more consistent we could define in base

Base.cconvert{N,T}(::Type{Ref{T}}, t::NTuple{N,T}) = Ref{NTuple{N,T}}(t)
Base.unsafe_convert{N,T}(::Type{Ref{T}}, r::Ref{NTuple{N,T}}) = unsafe_convert(Ref{NTuple{N,T}}, r)
1 Like
3

Thanks! That helps a lot. And to conclude, the trick is to just stick to Ref{Void}. (and as you’ve pointed out, I seem to have placed a few typos while reducing my code to the above example)

4

Too bad, I was too optimistic.
A minimal example fails to compile: cannot convert from Tuple{…}.

immutable MyType
    a::Int64
    MyType(v) = new(Int64(v))
    MyType() = new(zero(Int64))
end

function apicall(fn::Symbol, args...)    
    wargs = map(MyType, args)
    rarg = Ref(MyType())

    ccall(:apifn, Int64, 
        (Cstring, Int64, Ref{Void}, Ref{MyType}),
        string(fn), length(args), Ref(wargs), rarg
    ) != 0 && error("Call failed.")
end

apicall(:test, 1, 2, 3)

Giving the error

LoadError: MethodError: Cannot `convert` an object of type Base.RefValue{Tuple{MyType,MyType,MyType}} to an object of type Void
This may have arisen from a call to the constructor Void(...),
since type constructors fall back to convert methods.
while loading In[2], in expression starting on line 17

 in convert(::Type{Ref{Void}}, ::Base.RefValue{Tuple{MyType,MyType,MyType}}) at refpointer.jl:37
 in apicall(::Symbol, ::Int64, ::Vararg{Int64,N}) at In[2]:11

I remember having seen this error earlier this morning. I think that was when I switched to the previous pointer_from_objref solution…
Tried it with both Julia 0.5 and 0.6-dev from last week.

5

Sorry there was a typo. The Ref{Void} should be Ptr{Void}…

6

That works.
However, I figured it also works with a Ref{...} and the actual tuple when using a type alias:

immutable MyType
    a::Int64
    MyType(v) = new(Int64(v))
    MyType() = new(zero(Int64))
end

typealias MyTypeArgs{N} NTuple{N, MyType}

function apicall(fn::Symbol, args...; arguments = Tuple{})    
    wargs = map(MyType, args)
    rarg  = Ref(MyType())
    
    ccall(:apifn, Int64, 
        (Cstring, Int64, Ref{MyTypeArgs}, Ref{MyType}),
        string(fn), length(args), wargs, rarg
    ) != 0 && error("Call failed.")
end

apicall(:test, 1, 2, 3)

Anyway, case closed. Thanks.