CBinding.jl - bringing C's nested structs/unions, alignment, bitfields, variadic functions to Julia

I am announcing the availability of package CBinding.jl as a much improved way of interfacing C APIs from Julia with syntax analogous to the C you are binding to!

julia> using CBinding
julia> @cstruct MyCStruct {    # struct MyCStruct {
           c::Cchar            #     char c;
           i::Cint             #     int i;
           @cunion {           #     union {
               f::Cfloat       #         float f;
               d::Cdouble      #         double d;
           }                   #     };
       }                       # };
julia> @cstruct BitfieldStruct {    # struct BitfieldStruct {
           i::Cint:2                #     int i:2;
           j::Cuint:2               #     unsigned int j:2;
       }                            # };

julia> lib = Clibrary()  # dlopens the Julia process
Clibrary(Ptr{Nothing} @0x000061eefd6a1000)

julia> func = Cfunction{Cint, Tuple{Cstring, Vararg}}(lib, :printf)    # int printf(char *, ...)
Ptr{Cfunction{Nothing,Tuple{Cstring,Vararg{Any,N} where N}}} @0x000061eefc388930

julia> func("%s i%c %ld great demo of CBinding.jl v%3.1lf%c\n", "this", 's', 1, 0.1, '!')
this is 1 great demo of CBinding.jl v0.1!

I’m looking to get community feedback on how well it works on various platforms (only tested on Linux x86_64 to this point) and what syntax or package improvements might be desired. If you interface with C libraries from Julia, then please check out this package and help make it a great addition to the ecosystem! Thank you!

26 Likes

Hi,

Thank you for creating such a wonderful package. I always have to decode some binary data in a binary file and sometimes they use packed alignment or bitfield. So I wonder if CBinding will enable something like this in the futures.

unsafe_wrap for @cstruct, that would be great if I can do that, right now I know unsafe_load works for @cstruct but I think that will involve memory copy and will be slower. It will be great if unsafe_wrap or reinterpret works for @cstruct

Thank you

1 Like

That is a great suggestion! Just to clarify, you would like to take a Vector{Int8} and “cast” it into a @cstruct or have something like read(io, Cstruct) to directly create the @cstruct from a stream?

Either way works. If you can do the case from int8 vector to c struct then the direct read from io will be easy.

I haven’t read thru ur code but may I ask internally is cstruct actually have the same layout as C? Or is it just normal unpacked Julia struct with some convert function.

It will be very very great if it is indeed the same layout.

Right now I have to write my c library to handle this import…

It is the same layout as C as far as I can tell, and there are over 3,000 tests that cover different packing and alignment including bitfields.

It works by interpreting a block of memory in the same way that C interprets it, yet accessing primitive fields in a Cstruct has the same performance as if it is a Julia mutable struct and access within a Cconst(Cstruct) is the same performance as a Julia immutable struct.

if you already have the same C layout I think it will be very easy since you already have the _pointer() function.

Right now I ran in to segmentation fault if I try to do the following

using CBinding
@cstruct StructA {
    a::Cint
    b::Cchar
} __packed__

r = rand(UInt8, 5)
r_ptr = pointer(r)

ptr = reinterpret(Ptr{StructA}, r_ptr)
unsafe_wrap(Array, ptr, 1)

There is a difference between how mutable and immutable structs can be used in Julia.

julia> struct S
           a::Cint
       end

julia> r = rand(UInt8, sizeof(S))
4-element Array{UInt8,1}:
 0xc5
 0x5b
 0x3b
 0x1d

julia> r_ptr = pointer(r)
Ptr{UInt8} @0x00007fecc2a21af0

julia> ptr = reinterpret(Ptr{S}, r_ptr)
Ptr{S} @0x00007fecc2a21af0

julia> unsafe_wrap(Array, ptr, 1)
1-element Array{S,1}:
 S(490429381)

julia> mutable struct M
           a::Cint
       end

julia> ptr = reinterpret(Ptr{M}, r_ptr)
Ptr{M} @0x00007fecc2a21af0

julia> unsafe_wrap(Array, ptr, 1)
signal (11): Segmentation fault

You can use ptr = reinterpret(Ptr{Cconst(StructA)}, r_ptr) to work with the immutable form of StructA or use unsafe_load(ptr) instead of unsafe_wrap to create a (copied) mutable StructA. Please connect with me on julialang.slack.com for more help.

I should also mention that you could use a Caccessor to directly operate on the array memory (probably very similar to what unsafe_wrap would do actually).

julia> s=Caccessor{StructA}(ptr)
StructA(a=292813351, b=23)

julia> s.a = 42
42

julia> s.b = 0
0

julia> r
5-element Array{UInt8,1}:
 0x2a
 0x00
 0x00
 0x00
 0x00

work like a charm…

best Julia package I found so far…

1 Like

Thanks for a very interesting package! I have problems in calling a C function (new_foo in the MWE below) from a wrapped library with an enum in arguments. Could you give a simple example how to do this?

As a pseudo-MWE when I call

libfoo.Cbinding_new_foo("filename", 0.0, 0, 1, C_NULL, c"PROBLEMATIC_ENUM")

I get

ERROR: MethodError: no method matching Main.libfoo.Cbinding_new_foo(::String, ::Float64, ::Int64, ::Int64, ::Ptr{Nothing}, ::var"(c\"Stokes\")")
Stacktrace:
 [1] top-level scope
   @ REPL[28]:1

with the definition

typedef enum {
  PROBLEMATIC_ENUM=0,
....
} Stokes;
julia> libdifmap.c"PROBLEMATIC_ENUM"
var"(c\"Stokes\")" (0)
....
->c"PROBLEMATIC_ENUM" (0)
....

Thanks!

If you generated bindings for a function called new_foo, then you should be calling new_foo (or c"new_foo") from Julia. Cbinding_new_foo is an implementation detail (a singleton helper type) that could change in future releases of CBinding.

julia> using CBinding

julia> c``

julia> c"""
           typedef enum {
               MY1 = 0,
               MY2,
               MY3
           } MyEnum;

           inline static void new_foo(char *str, double d, int i, int j, int *ptr, MyEnum e) { }
       """jw;

julia> new_foo("filename", 0.0, 0, 1, C_NULL, MY1)
2 Likes

Yes, c""" ... """jw did the trick, I did not read the docs well enough, sorry. It seems to be amazingly easy to make a wrapper with CBinding.jl. I’ll try the next step (communicating with the library with struct pointers :grimacing:). Many thanks!