Loading arrays from C calls

Internals & Design
1

Hi,
I need to work with packages that wrap a C-callable library (*.so). Code that worked previously now causes segfaults when trying to load arrays. As a test, I compiled a small library called retArray.so containing the following code: (the function getRandom() I is from here)

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

/* function to generate and return random numbers */
int * getRandom( ) {

   static int r[10];
   int i;

   /* set the seed */
   srand( (unsigned)time( NULL ) );
  
   for ( i = 0; i < 10; ++i) {
      r[i] = rand();
      printf( "r[%d] = %d\n", i, r[i]);
   }

   return r;
}

I then use ccall to use the function getRandom like so in Julia:

julia> retRndArr() = ccall(("getRandom", "/home/anton_hinneck/projects/ccall_test/retArray.so"), Ptr{Cint}, ())
       arr = retRndArr()
       unsafe_load(arr, 1)
r[0] = 1960730611
r[1] = 681308252
r[2] = 883636336
r[3] = 1796652726
r[4] = 1091657955
r[5] = 1256249065
r[6] = 2083403509
r[7] = 463289330
r[8] = 1841375418
r[9] = 1100901313
1960730611

The function unsafe_load(array, i) returns a single value in array at index i. Of course I could now declare an array in Julia and assign values. However, how do I interpret data that starts at memory address arr as an Array{Cint, 1} (or Vector{Cint} for that matter) with n values?

Edit: unsafe_wrap(Vector{Cint}, arr, 10) achieves that.

Best, Anton

2

Look into the function unsafe_wrap.

2 Likes
3

Thanks, works.

4

Consider int* r = malloc(sizeof(int) * 10); and subsequent unsafe_wrap(Array, p::Ptr{Cint}, 10; own = true).

5

I tracked down the problem that inspired this post initially. I changed the sample code in my library to:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

/* function to generate and return random numbers */
int getRandom( int *r ) {

   int i;

   /* set the seed */
   srand( (unsigned)time( NULL ) );
  
   for ( i = 0; i < 10; ++i) {
      r[i] = rand();
      printf( "r[%d] = %d\n", i, r[i]);
   }

   return 0;
}

// int main(){
//     int r[10];
//     getRandom(r);
//     printf("First array value: %d \n",r[0]);
//     printf("Last array value: %d \n",r[9]);
//     return 0;
// }

Now calling the following in Julia results in a segfault:

result = Ptr{Cint}()
ccall(("getRandom", "/home/anton_hinneck/projects/ccall_test/retArray.so"), Ptr{Cint}, (Ptr{Cint},), result)
unsafe_wrap(Vector{Cint}, pointer_from_objref(result), 10)

What can I do to fix that? It can always be assumed that I know length and datatype of the array.

6

Thanks a lot. Yes, there is something about memory allocation that is either bugged or I didn’t wrap my head around properly.

7

Ptr{Cint}() will not actually allocate an array, it’s just a null pointer. What you probably want to do is create a Julia array and pass a pointer to that array to your c function, making sure that the array isn’t cleaned up by Julia’s GC before the ccall is done. Try:

result = Array{Cint}(undef, 10)
GC.@preserve result begin
    ccall(("getRandom", "/home/anton_hinneck/projects/ccall_test/retArray.so"), Cint, (Ptr{Cint},), result)
end
result
1 Like
8

That looks unnecessarily defensive. From Calling C and Fortran Code · The Julia Language

ccall automatically arranges that all of its arguments will be preserved from garbage collection until the call returns.

2 Likes
9

True, in this case you won’t need GC.@preserve, but it’s important to keep in mind for more complicated examples.

10

Simeon, thanks a lot.

11

The main issue was with your C code: int r[10]; does not allocate int[10] on the heap, it allocates it on the stack. This means that its lifetime ends once the function returns – so returning a ponter to r is a sure way to use-after-free. So you need to malloc the thing on the heap.

Now since you allocated something on the unmanaged C heap, you better figure out when to free it, or be sure that this memory leak is OK for your application. You can do this manually by calling free from C, or manually by calling ccall(:free, ... from julia.

Alternatively, you can promise the julia runtime that nobody else holds a pointer to the malloced memory, via own = true. Then the managed julia runtime takes possession of the allocation, freeing it when the array gets garbage collected, potentially realloced when you push! into the array, etc.

4 Likes