Accessing fields of structs passed from C to Julia

I’m using PackageCompiler to create a C-compatible library. The library’s main purpose is to analyze data that’s collected and passed from an existing C codebase. The communication between my library and the existing code happens via a handful of interface functions; these functions take pointers to structs as arguments, read data from the structs, and process accordingly.

However, I’ve run into an issue with accessing the fields of the structs.
Below is a MWE that compiles, but segfaults.

// mylib.h
#include "julia.h"
#include "uv.h"

typedef struct MyStruct {
    float cy;
    float cx;
} MyStruct;

float getcx(MyStruct* ms);
# MyLib.jl
module MyLib

export MyStruct, getcx

struct MyStruct
    cx::Cfloat
    cy::Cfloat
end

Base.@ccallable function getcx(msp::Ptr{MyStruct})::Cfloat
    ms = unsafe_load(msp)
    return ms.cx
end

end
// test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    MyStruct* ms;
    ms->cx = 0.0;
    ms->cy = 0.0;
    float cx = getcx(ms);
    return 0;
}

I’ve reviewed the documentation regarding mapping C types to julia, so I think the problem lies in the loading of Ptr{MyStruct}. Is this the case? Is there some detail of managing the memory or defining the structs that I’ve missed or misunderstood?

I suppose that in this case, a minimal working example should include the build process, as well. I’m following the structure of the MyLib example from PackageCompiler. That is, the directory structure is

── Makefile
├── Manifest.toml
├── Project.toml
├── build
│   ├── Manifest.toml
│   ├── Project.toml
│   ├── additional_precompile.jl
│   ├── build.jl
│   ├── generate_precompile.jl
│   └── mylib.h
├── src
│   └── MyLib.jl
└── test
    ├── Makefile
    ├── test.c

The MyLib.jl/build/build.jl file contains:

# MyLib.jl/build/build.jl
using PackageCompiler

target_dir = get(ENV, "OUTDIR", "$(@__DIR__)/target")
target_dir = replace(target_dir, "\\" => "/")       # Change Windows paths to use "/"

println("Creating library in $target_dir")
PackageCompiler.create_library(
    ".",
    target_dir;
    lib_name="mylib",
    precompile_execution_file=["$(@__DIR__)/generate_precompile.jl"],
    precompile_statements_file=["$(@__DIR__)/additional_precompile.jl"],
    incremental=true,
    force=true,
    filter_stdlibs=true,
    header_files=["$(@__DIR__)/mylib.h"],
)

The generate_precompile.jl and additional_precompile.jl files are both empty. As for the two makefiles, the top-level MyLib.jl/Makefile was adapted from the MyLib example:

# MyLib.jl/Makefile

JULIA ?= julia
DLEXT := $(shell $(JULIA) --startup-file=no -e 'using Libdl; print(Libdl.dlext)')

TARGET="./build/target"

MYLIB_INCLUDES = $(TARGET)/include/julia_init.h $(TARGET)/include/mylib.h
MYLIB_PATH := $(TARGET)/lib/libmylib.$(DLEXT)

$(MYLIB_PATH) $(MYLIB_INCLUDES): build/build.jl src/MyLib.jl
	$(JULIA) --startup-file=no --project=. -e 'using Pkg; Pkg.instantiate()'
	$(JULIA) --startup-file=no --project=build -e 'using Pkg; Pkg.instantiate(); include("build/build.jl")'

.PHONY: clean
clean:
	$(RM) *~ *.o *.$(DLEXT)
	$(RM) -Rf $(TARGET)

The makefile MyLib.jl/test/Makefile used to build the test C program was adapted from libcg:

ifeq ($(OS),Windows_NT)     # is Windows_NT on XP, 2000, 7, Vista, 10...
    OS := Windows
else
    OS := $(shell uname)
endif

ROOT_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))

JULIA ?= julia
JULIA_DIR := $(shell $(JULIA) --startup-file=no -e 'print(dirname(Sys.BINDIR))')
DLEXT := $(shell $(JULIA) --startup-file=no -e 'using Libdl; print(Libdl.dlext)')

PREFIX ?= $(ROOT_DIR)/../build/target
BINDIR := $(PREFIX)/bin
INCLUDE_DIR = $(PREFIX)/include
LIBDIR := $(PREFIX)/lib
TEST := test
LIBMYLIB := libmylib.$(DLEXT)

ifeq ($(OS), Windows)
  LIBDIR := $(BINDIR)
  TEST := test.exe
endif

LIBMYLIB_INCLUDES = $(INCLUDE_DIR)/julia_init.h $(INCLUDE_DIR)/mylib.h
LIBMYLIB_PATH := $(LIBDIR)/$(LIBMYLIB)

.DEFAULT_GOAL := $(TEST)

ifneq ($(OS), Windows)
  WLARGS := -Wl,-rpath,"$(LIBDIR)" -Wl,-rpath,"$(LIBDIR)/julia"
endif

CFLAGS+=-O2 -fPIE -I$(JULIA_DIR)/include/julia -I$(INCLUDE_DIR)
LDFLAGS+=-lm -L$(LIBDIR) -ljulia $(WLARGS)

test.o: test.c $(LIBMYLIB_INCLUDES)
	$(CC) $< -c -o $@ $(CFLAGS)

$(TEST): test.o
	mkdir -p $(BINDIR) 2>&1 > /dev/null
	$(CC) -o $@ $< $(LDFLAGS) -lmylib

.PHONY: install
install: $(TEST)
	cp $(TEST) $(PREFIX)/bin

.PHONY: clean
clean:
	$(RM) *~ *.o *.$(DLEXT) $(TEST)

this might help

1 Like

Thanks! I’ll take a look through the examples there and see if any of them help clear up my confusion.

You are not calling init_julia(argc, argv) at the beginning of main or shutdown_julia(0) at the end?

1 Like

Thanks, calls to init_julia and shutdown_julia were indeed part of the actual code that I forgot to include in the example. After adding them, the example above compiles, runs, and returns the proper result.

Updating the example to more closely reflect the actual library:

# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner

const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::Ptr{Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arrp = inner.arr
        GC.@preserve arrp begin
            arr = unsafe_wrap(Array, arrp, inner.n)
            return arr[1]
        end
    end
end

end
// MyLib.jl/build/mylib.h
#include "julia.h"
#include "uv.h"

#define ARRLENGTH 5

typedef struct MyInnerStruct {
    int n;
    float arr[ARRLENGTH];
} MyInnerStruct;

typedef struct MyOuterStruct {
    MyInnerStruct inner;
} MyOuterStruct;

float get_inner_first_val(MyOuterStruct* ms);
// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyInnerStruct inner;
    inner.n = 3;
    for (int i = 0; i < inner.n; i++) {
        inner.arr[i] = 1.0;
    }

    MyOuterStruct outer;
    outer.inner = inner;

    float v = get_inner_first_val(&outer);

    printf("%f\n", v);

    shutdown_julia(0);
    return 0;
}

That is, MyOuterStruct has a single field inner::MyInnerStruct, and MyInnerStruct has two fields: a fixed size array arr and its size n. This mirrors the full library; all of the arrays are of fixed size, but the actual useful data might not take up the full array.

Running test now throws the following error:

~/MyLib/test » ./test                                                   ubuntu@ip-172-31-47-177

signal (11): Segmentation fault
in expression starting at none:0
getindex at ./array.jl:861 [inlined]
get_inner_first_val at /home/ubuntu/MyLib/src/MyLib.jl:23
unknown function (ip: 0x7f63500800d5)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
get_inner_first_val at /home/ubuntu/MyLib/test/../build/target/lib/libmylib.so (unknown line)
main at /home/ubuntu/MyLib/test/test.c:19
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at ./test (unknown line)
Allocations: 2737 (Pool: 2728; Big: 9); GC: 0
[1]    1818958 segmentation fault (core dumped)  ./test

The line number referred to in this error corresponds to the return arr[1] line in MyLib.jl/src/MyLib.jl.

I believe this should be

const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::NTuple{ARRLENGTH, Cfloat}
end

since Ptr{Cfloat} to julia is a pointer to a single float, not a number of floats and C (if I recall correctly) stores that “array” inline in your struct. Haven’t checked though. So what would happen is that C just writes its floats and julia then tries to dereference them, which segfaults because an inline float is not a pointer.

Thanks. It appears you’re right. I changed the MyLib.jl file to read as follows:

# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner

const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::NTuple{ARRLENGTH,Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arr = inner.arr
        return arr[1]
    end
end

end

The rest of the code (the build files and the C test program) is the same. After re-compiling the library with PackageCompiler and re-compiling/linking the test.c file, the get_inner_first_val function now works as expected:

~/MyLib/test » ./test                                                                                                                                                                ubuntu@ip-172-31-47-177
1.000000

Unfortunately, the size of the array member in the real code is much, much larger; the equivalent of ARRLENGTH is currently set to 26214400 (the array member is meant to store image data, and the images are all quite large).

Updating ARRLENGTH to match the production value in mylib.h and MyLib.jl, with no other changes, test.c segfaults. Of course, in test.c the MyOuterStruct is allocated on the stack. I updated test.c to instead allocate the MyOuterStruct on the heap:

// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyOuterStruct* outer = malloc(sizeof(MyOuterStruct));

    outer->inner.n = 3;
    for (int i = 0; i < outer->inner.n; i++) {
        outer->inner.arr[i] = 1.0;
    }

    float v = get_inner_first_val(outer);

    printf("%f\n", v);

    free(outer);

    shutdown_julia(0);
    return 0;
}

This works, succesfully printing the first value stored in inner.arr. However, this takes a very long time (several minutes) to execute. That kind of execution speed is totally unacceptable for the context my library is meant to be used in (images should be loaded and analyzed in <1s.

My thought here was this long runtime this was related to JITing during the execution of the test program, so I added some statements to the pre-compilation file:

# MyLib.jl/build/generate_precompile.jl
using MyLib

arr = Tuple(zeros(Cfloat, ARRLENGTH));
inner = MyInnerStruct(zero(Cfloat), arr)
outer = MyOuterStruct(inner)

outer_ref = Ref(outer)

GC.@preserve outer_ref begin
    outer_p = Base.unsafe_convert(Ptr{MyOuterStruct}, outer_ref)
    cx = get_inner_first_val(outer_p)
end

However, the construction of inner here throws a StackOverFlowError, halting the compilation of the library.
This makes sense, since Tuples are not intended to store this many elements.

On the other hand, if I switch to a regular float* arr member in the C code and Ptr{Cfloat} in Julia, the whole program runs in about 0.2s. For completeness, here are the updated MyLib.jl, mylib.h, and test.c:

# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner_first_val, ARRLENGTH

const ARRLENGTH = 5120^2

struct MyInnerStruct
    n::Cint
    arr::Ptr{Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arrp = inner.arr
        GC.@preserve arrp begin
            arr = unsafe_wrap(Array, arrp, inner.n)
            return arr[1]
        end
    end
end

end
// MyLib.jl/build/mylib.h
#include "julia.h"
#include "uv.h"

#define ARRLENGTH 5120 * 5120

typedef struct MyInnerStruct {
    int n;
    float* arr;
} MyInnerStruct;

typedef struct MyOuterStruct {
    MyInnerStruct inner;
} MyOuterStruct;

float get_inner_first_val(MyOuterStruct* ms);
// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyOuterStruct* outer = malloc(sizeof(MyOuterStruct));

    float* arr = calloc(ARRLENGTH, sizeof(float));
    outer->inner.arr = arr;

    outer->inner.n = 100;
    for (int i = 0; i < outer->inner.n; i++) {
        outer->inner.arr[i] = 1.0;
    }

    float v = get_inner_first_val(outer);

    printf("%f\n", v);

    free(outer);

    shutdown_julia(0);
    return 0;
}

It seems like I’ve run into a fundamental design problem here; trying to pass fixed-size array members of the required size between Julia and C doesn’t seem to be feasible. I don’t have direct control over the C code that will call my library, but I could perhaps suggest some alterations to allow this set-up to work.

I’m a little bit sceptical about GCC putting such a large struct on the stack in the first place - it may be that it begins to box such large fields anyway. You’ll have to check with sizeof in C which happens for your real struct. The docs do suggest using NTuple anyway, but better safe than sorry I’d say.

That’s very surprising to me - can you run this under e.g. pprof to see where the time is spent? The tuple may be large, sure, but a simple access of the first index definitely shouldn’t take multiple minutes to even compile.

Well to me having such a large struct (even in C) is already pretty unusual - having all that data stored inline seems non-ideal cache-wise, as iterating over e.g. an array of them will have a huge number of wasted space. The dereference to a malloced array seems vastly insignificant compared to that.

I will indeed do some profiling to see what’s going on; it doesn’t really make sense that it would take so long. Since Julia won’t even allow a MyInnerStruct with such a large Tuple member to even be constructed without throwing a StackOverflowError, but constructing such a struct on the C side and passing it in at least works (if slow), I suspect you’re right that there’s some kind of boxing or something happening.

I agree that defining the C struct in such a way is a strange choice; it’s not one that was made by me, but it is one that I’m stuck with unless the maintainers of the C code are willing to make a modification.

I appreciate your help.

1 Like