# Ccall with a C struct containing a pointer

**URL:** <https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036>\
**Category:** General Usage\
**Created:** [December 20, 2024, 6:26pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036 "2024-12-20T18:26:19Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![luke-kiernan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luke-kiernan/32/214309_2.png) [@luke-kiernan](https://discourse.julialang.org/u/luke-kiernan)\
**Post date:** [December 20, 2024, 6:26pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/1 "2024-12-20T18:26:19Z")

</div>

I’m writing a Julia interface for a C library. I have a C struct containing C-style array. I’ve wrapped said struct in Julia like so:

```julia
mutable struct Example
   data::Ptr{Int64} # in the C, this is an array.
end
x = Int64[1,2,3]
s = Example(pointer(x))
@ccall lib.someFunc(s::Example)::Cvoid #this works as desired

```

How do I create a Julia class to encapsulate this functionality? Writing functions that take the type `Example` as an argument seems ripe for memory issues: what if the array that `data` points to goes out of scope?

In C++, I’d have `data` point to an `new`ed array allocated on the heap, which I then would `delete` at some point. From the docs on wrapping C and Fortran code in Julia, if I were calling a C function that took a C-style array, I would construct a `myArray` of type `Vector{T}` then pass `myArray::Ref{T}` into the `@ccall`. But the interface of the C function requires the struct: I cannot pass a bare array into the C function.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 20, 2024, 6:55pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/2 "2024-12-20T18:55:31Z")

</div>

> [@luke-kiernan](#):
>
> Writing functions that take the type `Example` as an argument seems ripe for memory issues: what if the array that `data` points to goes out of scope?

One option is to manually root the array during the `@ccall`, to make sure that it stays allocated until the call is complete, e.g.

```julia
GC.@preserve x @ccall lib.someFunc(s::Example)::Cvoid

```

Another option is to pad your `struct` on the Julia side with additional fields that store references to the Julia arrays, e.g.

```julia
mutable struct Example
    data::Ptr{Int64} # in the C, this is an array.
    _data::Vector{Int64} # keep a ref to the Julia array
    Example(d::Vector{Int64}) = new(pointer(d), d)
end

```

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 20, 2024, 7:59pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/3 "2024-12-20T19:59:38Z")

</div>

I think there’s a pattern involving `cconvert` and `unsafe_convert` here, with two structs. One for using on the Julia side, one for the actual FFI interop, which would only exist for the duration of the FFI call. I’m not 100% sure on the exact definitions necessary, so the following may not necessarily be correct:

```C
// The C example just returns the first element of the array

typedef struct {
  int* data;
} CExample;

int printarr(CExample* ex) {
  return ex->data[0];
}

```

```julia
mutable struct JLExample
    data::Vector{Int64}
end

mutable struct CExample
    data::Ptr{Int64}
end

Base.cconvert(::Type{Ptr{CExample}}, jle::JLExample) = CExample(pointer(jle.data))

function Base.unsafe_convert(::Type{Ptr{CExample}}, ce::CExample)
    return convert(Ptr{CExample}, pointer_from_objref(ce))
end

```

The reason I’m not 100% sure on this is because of that `pointer(jle.data)` in `cconvert`. This is likely illegal - `cconvert` isn’t supposed to take pointers, that’s the job of `unsafe_convert` here. The manual seems self contradictory here, since it says this:

> [`Base.cconvert`](https://docs.julialang.org/en/v1/base/c/#Base.cconvert) normally just calls [`convert`](https://docs.julialang.org/en/v1/base/base/#Base.convert), but can be defined to return an arbitrary new object more appropriate for passing to C. This should be used to perform all allocations of memory that will be accessed by the C code. For example, this is used to convert an `Array` of objects (e.g. strings) to an array of pointers.

but also this:

> This can be used to allocate memory that will be accessed by the `ccall`. If multiple objects need to be allocated, a tuple of the objects can be used as return value.
> 
> Neither `convert` nor `cconvert` should take a Julia object and turn it into a `Ptr`.

which seems to imply that the conversion of an `Array` of objects to an array of pointers would be illegal…

Either way, it would be used like so:

```julia
julia> function bar()
           ex = JLExample([15,2,3])
           GC.@preserve ex begin
               @ccall "./example.so".printarr(ex::Ref{CExample})::Cint
           end
       end
bar (generic function with 1 method)

julia> bar()
15

```

which works, but certainly feels a little ugly due to the need for that `GC.@preserve`. The `GC.@preserve` should make the definitions above safe, since `ex` (and by extension, the wrapped array) is not allowed to be freed until the end of that `begin` block. This shouldn’t be necessary though, and I’m pretty sure I’m missing the correct way of doing this 🤔 Would be really great to get such an example involving a more complicated struct into the docs…

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 20, 2024, 8:21pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/4 "2024-12-20T20:21:44Z")

</div>

Here’s one other variation that _might_ be more appropriate, and could be safer by default, if I’m using the interface correctly:

```julia
mutable struct JLExample
    data::Vector{Int64}
end

struct CExample # not mutable!
    data::Ptr{Int64}
end

Base.cconvert(::Type{Ref{CExample}}, jle::JLExample) = (jle, Ref{CExample}())

function Base.unsafe_convert(::Type{Ref{CExample}}, ce::Tuple{JLExample, Base.RefValue{CExample}})
    jle, ref = ce
    ref[] = CExample(pointer(jle.data))
    return Base.unsafe_convert(Ptr{CExample}, ref)
end

```

Basically, this ensures the C-FFI-struct has a safe place to go, by making sure both the source object `jle` and the inline data is preserved. This should be the case, since the [manual says](https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/#automatic-type-conversion) that the result of `cconvert` will “behave as if passed to `GC.@preserve`”. Then, in `unsafe_convert` the actual construction of the immutable data being passed to C is performed. Since that object is stored in the already allocated `Ref` (and is itself `isbits`, so will be stored inline in the `Ref`), all that’s left to do is convert that `Ref` to a `Ptr`, which will be passed to the `ccall`. (I’m not exactly sure why this last `unsafe_convert` is needed, but without it Julia complained about expecting a `Ptr`…)

This also seems to be zero-overhead on the calling side:

```julia
julia> ex = JLExample([15,2,3])
JLExample([15, 2, 3])

julia> function bar(ex)
           @ccall "./example.so".printarr(ex::Ref{CExample})::Cint
       end
bar (generic function with 1 method)

julia> bar(ex)
15

# no allocations!!
julia> @time bar(ex)
  0.000002 seconds
15

```

* * *

I should also mention that both of my posts assume that the original memory is allocated & managed by Julia. If the data is instead allocated & managed by C, the dance is going to be a bit different, I think.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 20, 2024, 8:31pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/5 "2024-12-20T20:31:34Z")

</div>

Yet a third variation that seems to work:

```julia
mutable struct JLExample
    data::Vector{Int64}
end

mutable struct CExample
    data::Ptr{Int64}
    CExample() = new()
end

Base.cconvert(::Type{Ref{CExample}}, jle::JLExample) = (jle, CExample())

function Base.unsafe_convert(::Type{Ref{CExample}}, ce::Tuple{JLExample, CExample})
    jle, ref = ce
    ref.data = pointer(jle.data)
    return convert(Ptr{CExample}, Base.pointer_from_objref(ref))
end

```

This is basically the same as the one above, except that `CExample` itself is `mutable`, instead of storing an immutable in a `Ref`. This way, we can take the pointer to our data in `unsafe_convert`, and just write that pointer into the existing struct, which is then passed to C. I think this one, as well as the previous one are equally good, but there’s still a little lingering uneasiness about the whole thing.

---

<div class="post-metadata">

**Author:** ![mattsignorelli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattsignorelli/32/221502_2.png) [@mattsignorelli](https://discourse.julialang.org/u/mattsignorelli)\
**Post date:** [December 20, 2024, 9:07pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/6 "2024-12-20T21:07:10Z")

</div>

In GTPSA.jl, we solved this problem by [allocating the array directly in C](https://github.com/bmad-sim/GTPSA.jl/blob/main/src/tps.jl#L17-L49). E.g. for a member array `coef`, we allocate in Julia using:

```julia
coef = @ccall jl_malloc(sz::Csize_t)::Ptr{T}

```

We then attach the finalizer to it:

```julia
f(t) = @ccall jl_free(t.coef::Ptr{Cvoid})::Cvoid
finalizer(f, t)

```

This solution has the benefits of keeping the C-style array owned by the C, safe from Julia’s garbage collector. I then pass to C functions like:

```julia
s = TPS() # mutable type like your example
@ccall lib.someFunc(s::Ref{TPS})::Cvoid

```

Of course, you must make sure that flexible array members are not used in the C code for this to work.

---

<div class="post-metadata">

**Author:** ![luke-kiernan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luke-kiernan/32/214309_2.png) [@luke-kiernan](https://discourse.julialang.org/u/luke-kiernan)\
**Post date:** [December 24, 2024, 4:38pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/7 "2024-12-24T16:38:51Z")

</div>

Thanks for bringing `cconvert` and `unsafe_convert` to my attention: using those seems like the cleanest way. Storing both the pointer and the pointed-to data could also work…but it feels kind of clunky. However, when I try to imitate the above examples, I’m getting an `Illegal instruction` error.

> [@Sukera](#):
>
> `int printarr(CExample* ex) {return ex->data[0];}`

One thing I should’ve mentioned: the C functions I’m interested in typically take the C struct itself, not a pointer to the C struct. In that case, do things look different than in the above examples?

More details about my goals: I’m trying to build a Julia interface for the Apple Accelerate routines related to sparse matrix factorization. (Related links: my work-in-progress/proof-of-concept [GitHub repo](https://github.com/luke-kiernan/AASparseSolvers.jl/tree/main) and [this forum post](https://discourse.julialang.org/t/apple-accelerate-sparse-solvers/110175).) I want to avoid re-inventing the wheel: ideally, `cconvert` should accept a `SparseMatrixCSC` (the Julia struct) and create a matching [`SparseMatrix`](https://github.com/luke-kiernan/AASparseSolvers.jl/blob/main/src/wrappers.jl#L101) struct (the C struct), for passing into the various `@ccall`s. The structs mostly line up, but there’s small differences: 0- vs 1-indexing, Cints vs Clongs for row indices.

For a more minimal, less involved example: take a look at [`DenseMatrix`](https://github.com/luke-kiernan/AASparseSolvers.jl/blob/main/src/wrappers.jl#L106). This only has a single pointer, and no re-indexing is required. How would I write `cconvert` so as to take a Julia `A::Matrix{T}` and create a `B::DenseMatrix{T}` such that `B.data = pointer(A.ref)`, so I could have [`SparseMultiply`](https://github.com/luke-kiernan/AASparseSolvers.jl/blob/main/src/wrappers.jl#L149) take in normal Julia matrices for `arg2` and `arg3`?

EDIT: okay I have a bigger problem. I’m nesting structs (`SparseMatrixStructure` inside `SparseMatrix`), where the inner struct is not an `isbits` struct type. Thus the `Illegal instruction` error: fields weren’t lining up. But I’d still be interested in knowing the answer to the above question.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [December 25, 2024, 9:25am UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/8 "2024-12-25T09:25:13Z")

</div>

> [@luke-kiernan](#):
>
> In that case, do things look different than in the above examples?

Not really - the difference is just that you wouldn’t pass a `Ptr`/`Ref` in the `ccall` but just the struct directly, without the need for `pointer_from_objref` and the like at all.

> [@luke-kiernan](#):
>
> The structs mostly line up, but there’s small differences: 0- vs 1-indexing, Cints vs Clongs for row indices.

If the size/type of the individual items in the struct is different, you’ll want/need to make sure to convert the data to that type. So from julian `Int64` to a `Cint`, by storing the data in a new bit of memory.

> [@luke-kiernan](#):
>
> For a more minimal, less involved example: take a look at [`DenseMatrix`](https://github.com/luke-kiernan/AASparseSolvers.jl/blob/main/src/wrappers.jl#L106). This only has a single pointer, and no re-indexing is required. How would I write `cconvert` so as to take a Julia `A::Matrix{T}` and create a `B::DenseMatrix{T}` such that `B.data = pointer(A.ref)`

That should be something like

```julia
function Base.cconvert(::Type{DenseMatrix{T}}, m::Matrix{T}) where T
    rowCount, colCount = size(m)
    stride = 1 # not 100% sure on this - check that row/column majorness aligns! Julia is column major!
    dm = DenseMatrix{T}(rowCount, colCount, strig, #= the attribute type object =#, C_NULL)
    return (m, dm)
end

function Base.unsafe_convert(::Type{DenseMatrix{T}}, tup::Tuple{Matrix{T}, DenseMatrix{T}}) where T
    m, dm = tup
    dm.data = pointer(m) # m is GC-preserved in this function, so this is safe
    dm
end

```

> [@luke-kiernan](#):
>
> I’m nesting structs (`SparseMatrixStructure` inside `SparseMatrix`), where the inner struct is not an `isbits` struct type. Thus the `Illegal instruction` error: fields weren’t lining up.

Yes, the types you end up passing to C must match _exactly_ in terms of layout. You can check `Base.allocated_inline`, which should help with that. The important thing is that the outermost struct you pass is mutable (either because the struct itself is mutable, or it’s a `Ref`, which is just a mutable struct with a single field), so that any modification C makes into that memory is visible again on the julia side. To Julia this would be “as if” the entire content was replaced, even though C just performed a partial write.

---

<div class="post-metadata">

**Author:** ![luke-kiernan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luke-kiernan/32/214309_2.png) [@luke-kiernan](https://discourse.julialang.org/u/luke-kiernan)\
**Post date:** [December 27, 2024, 3:55pm UTC](https://discourse.julialang.org/t/ccall-with-a-c-struct-containing-a-pointer/124036/9 "2024-12-27T15:55:02Z")

</div>

> [@Sukera](#):
>
> That should be something like
> 
> ```julia
> function Base.cconvert...
> 
> ```

Great! Thank you: answer accepted. I’ve expanded that snippet into a self-contained example, for posterity:

```julia
mutable struct DenseMatrix # layout matches the C.
    rowCount::Cint
    columnCount::Cint
    data::Ptr{Cdouble}
end
function Base.cconvert(::Type{DenseMatrix}, m::Matrix{Float64})
    return (m, DenseMatrix(size(m)[1], size(m)[2], C_NULL))
end
function Base.unsafe_convert(::Type{DenseMatrix}, tup::Tuple{Matrix{Float64}, DenseMatrix})
    m, dm = tup
    dm.data = pointer(m) # m is GC-preserved in this function, so this is safe
    return dm
end
function CWrapper(arg1::Matrix{Float64}) # Julia base type here...
    @ccall lib.fname(arg1::DenseMatrix) # ...and C struct type here.
end

```

It’d be good to get a more elaborate example like this in the docs.