Passing a reference to Julia struct into ccall and let C manage the memory

ccall
memory-allocation

#1

Hi Julia users,

I am having trouble with a ccall. Here my problem in a simplified form:

  1. Declare a struct MyArray in Julia that holds a pointer ‘data’ to a Cdouble array and two Cints for the size ‘m’ and ‘n’,

  2. Initialize a variable ‘jArray’ of type MyArray with m=n=-1 and data=C_NULL

  3. Initialize an array ‘sourceArray’ of with some Cdouble data

  4. Pass a reference of ‘sourceArray’, its size and a reference to the variable ‘jArray’ to ccall

  5. The C-function (of type void) ‘libfillarray’ sets the pointer in->data to point to the ‘sourceArray’

It is important that I let C manage the memory.

I am having trouble to understand why the address of the pointer changes. Does anyone have an explanation for this behavior? I probably misunderstood something…

Here is an example output (for the code scroll down):

*****************
Julia addresses (before ccall):
MyArray:                  Ptr{Void} @0x00007f09bb8a0c90
Data in MyArray:          Ptr{Float64} @0x0000000000000000
Size of MyArray in Julia: 16
Source array:             Ptr{Float64} @0x00007f09bc7cd8b0
*****************



_______________________________________
C addresses...
MyArray       : 0x7f09bb8a0f10   |   Array in MyArray: (nil)   |   Size of MyArray 'in' in C: 16
Source array  : 0x7f09bc7cd8b0

Filling array...

New C addresses...
MyArray       : 0x7f09bb8a0f10   |   Array in MyArray: 0x7f09bc7cd8b0   |   Size of MyArray 'in' in C: 16
Source array  : 0x7f09bc7cd8b0
_______________________________________



*****************
Julia addresses (after ccall):
MyArray:                  Ptr{Void} @0x00007f09bb8a0f50
Data in MyArray:          Ptr{Float64} @0x0000000000000000
Size of MyArray in Julia: 16
Source array:             Ptr{Float64} @0x00007f09bc7cd8b0

Here the code in Julia:

struct MyArray
    # Holds a pointer to an (m,n)-array
    m :: Cint
    n :: Cint

    data :: Ptr{Cdouble}

    # Constructor initializes empty Array
    MyArray() = new(-1,-1,C_NULL)
end


function writeArray(m :: T, n :: T) where {T<:Int}

    # Write array 'sourceArray' in MyArray.A
    
    # set up a Float64 (Cdouble) array
    sourceArray = reshape(collect(1:1.0:(n*m)),m,n)

    # initialize empty struct
    jArray = MyArray()

    # --------------------------------
    # Print addresses
    p_before = pointer_from_objref(jArray)
    p_array_before = jArray.data
    size_before = sizeof(jArray)



    println("\n\n\n*****************")
    println("Julia addresses (before ccall):")
    
    # Adress of julia struct
    println("MyArray:                  $p_before") 
    println("Array in MyArray:         $p_array_before") 
    println("Size of MyArray in Julia: $size_before")

    # Adress of 'sourceArray'
    println("Source array:             $(pointer(sourceArray))")
    println("*****************\n")
    # --------------------------------

    # Cann C library function that makes the array pointer in 'jArray' point
    # to the data stored in 'sourceArray'
    ccall((:fill_array, "libfillarray"), 
                        Void,
                        (Ref{MyArray}, 
                        Cint, Cint, Ref{Cdouble}),
                        Ref(jArray),
                        Cint(m), Cint(n), sourceArray)


    # --------------------------------
    # Print addresses after ccall
    p_after = pointer_from_objref(jArray)
    p_array_after = jArray.data
    size_after = sizeof(jArray)

    println("\n\n\n*****************")
    println("Julia addresses (after ccall):")
    
    # Adress of julia struct changed again (in ccall and here)
    println("MyArray:                  $p_after") 
    println("Array in MyArray:         $p_array_after") 
    println("Size of MyArray in Julia: $size_after")
    
    # Here no change in address
    println("Source array:             $(pointer(sourceArray))")
    
    println("*****************\n")
    # --------------------------------

    return jArray
end

Here the C-code:

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

struct MyArray {
    int m;
    int n;

    double *data;
};

typedef struct MyArray MyArray_t;


void fill_array(MyArray_t* in,
                int m, int n, double* sourceArray)
{
    int i, j;
    
    printf("\n\n_______________________________________\n");
    // Print out the address of struct pointer 'in', 'in->data' 
    printf("C addresses...\n");
    printf("MyArray       : %p   |   Array in MyArray: %p   |   Size of MyArray 'in' in C: %lu\n", in, in->data, sizeof(*in));
    printf("Source array  : %p\n\n", sourceArray);

    // Allocate space in for the array
    //printf("Allocating memory...\n\n");
    //in->data = (double* ) malloc(m * n * sizeof(double));

    // Let the array pointer in struct 'in' passed by Julia point to the array
    // 'sourceArray'
    printf("Filling array...\n\n");
    in->m = m;
    in->n = n;
    in->data = sourceArray;
    
    // Print out the address of struct pointer 'in', 'in->data' after
    // changing array pointer
    printf("New C addresses...\n");
    printf("MyArray       : %p   |   Array in MyArray: %p   |   Size of MyArray 'in' in C: %lu\n", in, in->data, sizeof(*in));
    printf("Source array  : %p", sourceArray);
    printf("\n_______________________________________\n");

}

#2

Try with mutable struct.

Also I think you allocate memory and then “throw” it away in the C function.

See also https://github.com/JuliaLang/julia/commit/1502bd0b436d2675d15a7df5ab02e8bb27622f59 on why you should not use pointer_from_objref on immutable values like your MyArray.


#3

Many thanks. Unfortunately this still happens with a mutable struct. I believe there is some memory management difference between julia and C. It seems like the structs are not equivalent (memory alignment?). However, I might be wrong or have misunderstood something in the documentation.

Also, I think I am aware of (some of) the problems with pointer_from_objref() and I am not using it in the ccall.


#4

It seems like the structs are not equivalent (memory alignment?)

What makes you think that? Your C-code for the size is wrong; you want sizeof(*in).

Alternatively to mutable struct, you could also use a struct and Ref (i.e.jArray = Ref(MyArray())).

Also, what at-thofma said. Your C code makes no sense at all; it mallocs and then leaks the memory.


#5

@thofma @foobar_lv2 Many thanks! The struct must indeed be mutable (didn’t see this because I forgot to reload the module). Problem solved! :slight_smile:

About the C-code you are of course right. It should be sizeof(*in) and the allocation was not intended to be there. But this was not the major Problem.