Creating an Array{Array{UInt8}} in C

Internals & Design Internals
1

I want to create an Array{Array{UInt8}} in C and wondered if I got all the garbage collection and write barriers right. The code below compiles and works.

#include <julia.h>

extern "C" {
  jl_value_t* test_array_array(int i) {
    jl_value_t *t = jl_apply_array_type(jl_apply_array_type((jl_value_t*)jl_uint8_type, 1), 1);
    jl_value_t *a = (jl_value_t*)jl_alloc_array_1d(t, i);
    JL_GC_PUSH1(&a);

    jl_array_t **d = (jl_array_t**)jl_array_data(a);

    for (int j = 0; j < i; j++) {
      jl_array_t *aa = jl_alloc_array_1d(jl_array_uint8_type, i);
      JL_GC_PUSH1(&aa);

      unsigned char *dd = (unsigned char*)jl_array_data(aa);

      for (int k = 0; k < i; k++) {
        dd[k] = (unsigned char)k;
      }

      d[j] = aa;
      jl_gc_wb(a, aa);

      JL_GC_POP(); //aa
    }

    JL_GC_POP(); //a

    return a;
  }
}

Call it with

g() = ccall((:test_array_array, LIB_PATH),
            Array{Array{UInt8}}, (Cint,), 5)
2

On a general note: What is the trade-off between returning Julia objects directly from C, as above, and writing a general C interface and calling that from Julia — besides the obvious that the latter can be used by other programming languages.

3

Writing a general C interface and calling that from Julia is the recommended approach. The other way requires writing and recompiling C code just to update the interface between Julia and C and depends on Julia internals which are not guaranteed to be stable. The ccall approach on the other hand is guaranteed to continue working.

1 Like
4

This is not needed.

Otherwise looks correct.

The question is why do you want to write the code in C. I’ve only seen two cases where this happens,

  1. You want to use some C functions that’s not available in julia.

    This could be header only information, threading (openmp for example) or other libraries.
    In most cases you won’t really need a julia specific interface since there’s usually little julia specific about the function you need in C. Some of these can also be done with Cxx.jl or other packages instead. In rare cases where a julia specific interface is simpler, you should be able to do that without much/any issue about API stability.

  2. You want to share code with other project / languages.

    In this case, you’ll likely want a C-only interface for obvious reason.

In general, I think in most cases having a julia specific interface will be harder to implement (the code you have above is much shorter with same/better performance in pure julia for example). If you find it simpler to interact with julia in C for the specific usecase you should feel free to do so. In these cases, it’ll be interesting to see why you couldn’t do that in nearly pure julia combined with ccall and llvmcall though and if such function actually makes sense to be included in julia.

2 Likes
5

Thanks for reading through the code. It is to interface some c++ code with julia that does not come with its own compiled library. I wanted to test the effort writing a general c interface and a julia interface. The julia part was a fun exercise to learn some of the internals but in the end the code for the c interface + the julia function to call it is shorter, easier to understand and more performant and guaranteed to be stable.