# Accessing fields of structs passed from C to Julia

**URL:** <https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712>\
**Category:** General Usage\
**Tags:** question, package, interoperability, package-compiler\
**Created:** [March 10, 2022, 6:45pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712 "2022-03-10T18:45:15Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 10, 2022, 6:45pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/1 "2022-03-10T18:45:15Z")

</div>

I’m using PackageCompiler to create a C-compatible library. The library’s main purpose is to analyze data that’s collected and passed from an existing C codebase. The communication between my library and the existing code happens via a handful of interface functions; these functions take pointers to `struct`s as arguments, read data from the `struct`s, and process accordingly.

However, I’ve run into an issue with accessing the fields of the `struct`s.  
Below is a MWE that compiles, but segfaults.

```nohighlight
// mylib.h
#include "julia.h"
#include "uv.h"

typedef struct MyStruct {
    float cy;
    float cx;
} MyStruct;

float getcx(MyStruct* ms);

```

```julia
# MyLib.jl
module MyLib

export MyStruct, getcx

struct MyStruct
    cx::Cfloat
    cy::Cfloat
end

Base.@ccallable function getcx(msp::Ptr{MyStruct})::Cfloat
    ms = unsafe_load(msp)
    return ms.cx
end

end

```

```nohighlight
// test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    MyStruct* ms;
    ms->cx = 0.0;
    ms->cy = 0.0;
    float cx = getcx(ms);
    return 0;
}

```

I’ve reviewed the [documentation](https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/#mapping-c-types-to-julia) regarding mapping C types to julia, so I think the problem lies in the loading of `Ptr{MyStruct}`. Is this the case? Is there some detail of managing the memory or defining the `struct`s that I’ve missed or misunderstood?

---

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 10, 2022, 7:08pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/2 "2022-03-10T19:08:56Z")

</div>

I suppose that in this case, a minimal working example should include the build process, as well. I’m following the structure of the [`MyLib` example](https://github.com/JuliaLang/PackageCompiler.jl/tree/master/examples/MyLib) from `PackageCompiler`. That is, the directory structure is

```julia
── Makefile
├── Manifest.toml
├── Project.toml
├── build
│ ├── Manifest.toml
│ ├── Project.toml
│ ├── additional_precompile.jl
│ ├── build.jl
│ ├── generate_precompile.jl
│ └── mylib.h
├── src
│ └── MyLib.jl
└── test
    ├── Makefile
    ├── test.c

```

The `MyLib.jl/build/build.jl` file contains:

```julia
# MyLib.jl/build/build.jl
using PackageCompiler

target_dir = get(ENV, "OUTDIR", "$(@ __DIR__ )/target")
target_dir = replace(target_dir, "\\" => "/") # Change Windows paths to use "/"

println("Creating library in $target_dir")
PackageCompiler.create_library(
    ".",
    target_dir;
    lib_name="mylib",
    precompile_execution_file=["$(@ __DIR__ )/generate_precompile.jl"],
    precompile_statements_file=["$(@ __DIR__ )/additional_precompile.jl"],
    incremental=true,
    force=true,
    filter_stdlibs=true,
    header_files=["$(@ __DIR__ )/mylib.h"],
)

```

The `generate_precompile.jl` and `additional_precompile.jl` files are both empty. As for the two makefiles, the top-level `MyLib.jl/Makefile` was adapted from the `MyLib` example:

```makefile
# MyLib.jl/Makefile

JULIA ?= julia
DLEXT := $(shell $(JULIA) --startup-file=no -e 'using Libdl; print(Libdl.dlext)')

TARGET="./build/target"

MYLIB_INCLUDES = $(TARGET)/include/julia_init.h $(TARGET)/include/mylib.h
MYLIB_PATH := $(TARGET)/lib/libmylib.$(DLEXT)

$(MYLIB_PATH) $(MYLIB_INCLUDES): build/build.jl src/MyLib.jl
	$(JULIA) --startup-file=no --project=. -e 'using Pkg; Pkg.instantiate()'
	$(JULIA) --startup-file=no --project=build -e 'using Pkg; Pkg.instantiate(); include("build/build.jl")'

.PHONY: clean
clean:
	$(RM) *~ *.o *.$(DLEXT)
	$(RM) -Rf $(TARGET)

```

The makefile `MyLib.jl/test/Makefile` used to build the test C program was adapted from [`libcg`](https://github.com/simonbyrne/libcg/blob/master/main-c/Makefile):

```makefile
ifeq ($(OS),Windows_NT) # is Windows_NT on XP, 2000, 7, Vista, 10...
    OS := Windows
else
    OS := $(shell uname)
endif

ROOT_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))

JULIA ?= julia
JULIA_DIR := $(shell $(JULIA) --startup-file=no -e 'print(dirname(Sys.BINDIR))')
DLEXT := $(shell $(JULIA) --startup-file=no -e 'using Libdl; print(Libdl.dlext)')

PREFIX ?= $(ROOT_DIR)/../build/target
BINDIR := $(PREFIX)/bin
INCLUDE_DIR = $(PREFIX)/include
LIBDIR := $(PREFIX)/lib
TEST := test
LIBMYLIB := libmylib.$(DLEXT)

ifeq ($(OS), Windows)
  LIBDIR := $(BINDIR)
  TEST := test.exe
endif

LIBMYLIB_INCLUDES = $(INCLUDE_DIR)/julia_init.h $(INCLUDE_DIR)/mylib.h
LIBMYLIB_PATH := $(LIBDIR)/$(LIBMYLIB)

.DEFAULT_GOAL := $(TEST)

ifneq ($(OS), Windows)
  WLARGS := -Wl,-rpath,"$(LIBDIR)" -Wl,-rpath,"$(LIBDIR)/julia"
endif

CFLAGS+=-O2 -fPIE -I$(JULIA_DIR)/include/julia -I$(INCLUDE_DIR)
LDFLAGS+=-lm -L$(LIBDIR) -ljulia $(WLARGS)

test.o: test.c $(LIBMYLIB_INCLUDES)
	$(CC) $< -c -o $@ $(CFLAGS)

$(TEST): test.o
	mkdir -p $(BINDIR) 2>&1 > /dev/null
	$(CC) -o $@ $< $(LDFLAGS) -lmylib

.PHONY: install
install: $(TEST)
	cp $(TEST) $(PREFIX)/bin

.PHONY: clean
clean:
	$(RM) *~ *.o *.$(DLEXT) $(TEST)

```

---

<div class="post-metadata">

**Author:** ![lawless-m](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lawless-m/32/30869_2.png) [@lawless-m](https://discourse.julialang.org/u/lawless-m)\
**Post date:** [March 10, 2022, 7:34pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/3 "2022-03-10T19:34:52Z")

</div>

this _might_ help

[https://github.com/lawless-m/Czoo.jl](https://github.com/lawless-m/Czoo.jl)

---

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 10, 2022, 8:52pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/4 "2022-03-10T20:52:18Z")

</div>

Thanks! I’ll take a look through the examples there and see if any of them help clear up my confusion.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 10, 2022, 9:22pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/5 "2022-03-10T21:22:02Z")

</div>

> [@acarta](#):
>
> Below is a MWE that compiles, but segfaults.

You are not calling `init_julia(argc, argv)` at the beginning of `main` or `shutdown_julia(0)` at the end?

---

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 10, 2022, 10:31pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/6 "2022-03-10T22:31:03Z")

</div>

Thanks, calls to `init_julia` and `shutdown_julia` were indeed part of the actual code that I forgot to include in the example. After adding them, the example above compiles, runs, and returns the proper result.

Updating the example to more closely reflect the actual library:

```julia
# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner

const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::Ptr{Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arrp = inner.arr
        GC.@preserve arrp begin
            arr = unsafe_wrap(Array, arrp, inner.n)
            return arr[1]
        end
    end
end

end

```

```cpp
// MyLib.jl/build/mylib.h
#include "julia.h"
#include "uv.h"

#define ARRLENGTH 5

typedef struct MyInnerStruct {
    int n;
    float arr[ARRLENGTH];
} MyInnerStruct;

typedef struct MyOuterStruct {
    MyInnerStruct inner;
} MyOuterStruct;

float get_inner_first_val(MyOuterStruct* ms);

```

```cpp
// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyInnerStruct inner;
    inner.n = 3;
    for (int i = 0; i < inner.n; i++) {
        inner.arr[i] = 1.0;
    }

    MyOuterStruct outer;
    outer.inner = inner;

    float v = get_inner_first_val(&outer);

    printf("%f\n", v);

    shutdown_julia(0);
    return 0;
}

```

That is, `MyOuterStruct` has a single field `inner::MyInnerStruct`, and `MyInnerStruct` has two fields: a fixed size array `arr` and its size `n`. This mirrors the full library; all of the arrays are of fixed size, but the actual useful data might not take up the full array.

Running `test` now throws the following error:

```bash
~/MyLib/test » ./test ubuntu@ip-172-31-47-177

signal (11): Segmentation fault
in expression starting at none:0
getindex at ./array.jl:861 [inlined]
get_inner_first_val at /home/ubuntu/MyLib/src/MyLib.jl:23
unknown function (ip: 0x7f63500800d5)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2247 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2429
get_inner_first_val at /home/ubuntu/MyLib/test/../build/target/lib/libmylib.so (unknown line)
main at /home/ubuntu/MyLib/test/test.c:19
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at ./test (unknown line)
Allocations: 2737 (Pool: 2728; Big: 9); GC: 0
[1] 1818958 segmentation fault (core dumped) ./test

```

The line number referred to in this error corresponds to the `return arr[1]` line in `MyLib.jl/src/MyLib.jl`.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 11, 2022, 8:17am UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/7 "2022-03-11T08:17:33Z")

</div>

> [@acarta](#):
>
> ```julia
> const ARRLENGTH = 5
> 
> struct MyInnerStruct
> n::Cint
> arr::Ptr{Cfloat}
> end
> 
> ```

I believe this should be

```julia
const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::NTuple{ARRLENGTH, Cfloat}
end

```

since `Ptr{Cfloat}` to julia is a pointer to a single float, not a number of floats and C (if I recall correctly) stores that “array” inline in your struct. Haven’t checked though. So what would happen is that C just writes its floats and julia then tries to dereference them, which segfaults because an inline float is not a pointer.

---

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 11, 2022, 5:29pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/8 "2022-03-11T17:29:13Z")

</div>

Thanks. It appears you’re right. I changed the `MyLib.jl` file to read as follows:

```julia
# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner

const ARRLENGTH = 5

struct MyInnerStruct
    n::Cint
    arr::NTuple{ARRLENGTH,Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arr = inner.arr
        return arr[1]
    end
end

end

```

The rest of the code (the build files and the C test program) is the same. After re-compiling the library with `PackageCompiler` and re-compiling/linking the `test.c` file, the `get_inner_first_val` function now works as expected:

```bash
~/MyLib/test » ./test ubuntu@ip-172-31-47-177
1.000000

```

Unfortunately, the size of the array member in the real code is much, much larger; the equivalent of `ARRLENGTH` is currently set to `26214400` (the array member is meant to store image data, and the images are all quite large).

Updating `ARRLENGTH` to match the production value in `mylib.h` and `MyLib.jl`, with no other changes, `test.c` segfaults. Of course, in `test.c` the `MyOuterStruct` is allocated on the stack. I updated `test.c` to instead allocate the `MyOuterStruct` on the heap:

```cpp
// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyOuterStruct* outer = malloc(sizeof(MyOuterStruct));

    outer->inner.n = 3;
    for (int i = 0; i < outer->inner.n; i++) {
        outer->inner.arr[i] = 1.0;
    }

    float v = get_inner_first_val(outer);

    printf("%f\n", v);

    free(outer);

    shutdown_julia(0);
    return 0;
}

```

This works, succesfully printing the first value stored in `inner.arr`. However, this takes a _very_ long time (several minutes) to execute. That kind of execution speed is totally unacceptable for the context my library is meant to be used in (images should be loaded and analyzed in `<1s`.

My thought here was this long runtime this was related to JITing during the execution of the test program, so I added some statements to the pre-compilation file:

```julia
# MyLib.jl/build/generate_precompile.jl
using MyLib

arr = Tuple(zeros(Cfloat, ARRLENGTH));
inner = MyInnerStruct(zero(Cfloat), arr)
outer = MyOuterStruct(inner)

outer_ref = Ref(outer)

GC.@preserve outer_ref begin
    outer_p = Base.unsafe_convert(Ptr{MyOuterStruct}, outer_ref)
    cx = get_inner_first_val(outer_p)
end

```

However, the construction of `inner` here throws a `StackOverFlowError`, halting the compilation of the library.  
This makes sense, since `Tuple`s are not intended to store this many elements.

On the other hand, if I switch to a regular `float* arr` member in the C code and `Ptr{Cfloat}` in Julia, the whole program runs in about 0.2s. For completeness, here are the updated `MyLib.jl`, `mylib.h`, and `test.c`:

```julia
# MyLib.jl/src/MyLib.jl
module MyLib

export MyOuterStruct, MyInnerStruct, get_inner_first_val, ARRLENGTH

const ARRLENGTH = 5120^2

struct MyInnerStruct
    n::Cint
    arr::Ptr{Cfloat}
end

struct MyOuterStruct
    inner::MyInnerStruct
end

Base.@ccallable function get_inner_first_val(msp::Ptr{MyOuterStruct})::Cfloat
    GC.@preserve msp begin
        ms = unsafe_load(msp)
        inner = ms.inner
        arrp = inner.arr
        GC.@preserve arrp begin
            arr = unsafe_wrap(Array, arrp, inner.n)
            return arr[1]
        end
    end
end

end

```

```cpp
// MyLib.jl/build/mylib.h
#include "julia.h"
#include "uv.h"

#define ARRLENGTH 5120 * 5120

typedef struct MyInnerStruct {
    int n;
    float* arr;
} MyInnerStruct;

typedef struct MyOuterStruct {
    MyInnerStruct inner;
} MyOuterStruct;

float get_inner_first_val(MyOuterStruct* ms);

```

```cpp
// MyLib.jl/test/test.c
#include <stdio.h>
#include <stdlib.h>

#include "julia_init.h"
#include "mylib.h"

int main(int argc, char const* argv[]) {
    init_julia(argc, argv);

    MyOuterStruct* outer = malloc(sizeof(MyOuterStruct));

    float* arr = calloc(ARRLENGTH, sizeof(float));
    outer->inner.arr = arr;

    outer->inner.n = 100;
    for (int i = 0; i < outer->inner.n; i++) {
        outer->inner.arr[i] = 1.0;
    }

    float v = get_inner_first_val(outer);

    printf("%f\n", v);

    free(outer);

    shutdown_julia(0);
    return 0;
}

```

It seems like I’ve run into a fundamental design problem here; trying to pass fixed-size array members of the required size between Julia and C doesn’t seem to be feasible. I don’t have direct control over the C code that will call my library, but I could perhaps suggest some alterations to allow this set-up to work.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 11, 2022, 5:49pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/9 "2022-03-11T17:49:26Z")

</div>

> [@acarta](#):
>
> Unfortunately, the size of the array member in the real code is much, much larger; the equivalent of `ARRLENGTH` is currently set to `26214400` (the array member is meant to store image data, and the images are all quite large).

I’m a little bit sceptical about GCC putting such a large struct on the stack in the first place - it may be that it begins to box such large fields anyway. You’ll have to check with `sizeof` in C which happens for your real struct. The docs do suggest using `NTuple` anyway, but better safe than sorry I’d say.

> [@acarta](#):
>
> This works, succesfully printing the first value stored in `inner.arr` . However, this takes a _very_ long time (several minutes) to execute. That kind of execution speed is totally unacceptable for the context my library is meant to be used in (images should be loaded and analyzed in `<1s` .

That’s very surprising to me - can you run this under e.g. pprof to see where the time is spent? The tuple may be large, sure, but a simple access of the first index definitely shouldn’t take multiple minutes to even compile.

> [@acarta](#):
>
> It seems like I’ve run into a fundamental design problem here; trying to pass fixed-size array members of the required size between Julia and C doesn’t seem to be feasible. I don’t have direct control over the C code that will call my library, but I could perhaps suggest some alterations to allow this set-up to work.

Well to me having such a large struct (even in C) is already pretty unusual - having all that data stored inline seems non-ideal cache-wise, as iterating over e.g. an array of them will have a huge number of wasted space. The dereference to a `malloc`ed array seems vastly insignificant compared to that.

---

<div class="post-metadata">

**Author:** ![acarta](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/acarta/32/28474_2.png) [@acarta](https://discourse.julialang.org/u/acarta)\
**Post date:** [March 11, 2022, 6:05pm UTC](https://discourse.julialang.org/t/accessing-fields-of-structs-passed-from-c-to-julia/77712/10 "2022-03-11T18:05:06Z")

</div>

I will indeed do some profiling to see what’s going on; it doesn’t really make sense that it would take _so_ long. Since Julia won’t even allow a `MyInnerStruct` with such a large `Tuple` member to even be constructed without throwing a `StackOverflowError`, but constructing such a struct on the C side and passing it in at least works (if slow), I suspect you’re right that there’s some kind of boxing or something happening.

I agree that defining the C struct in such a way is a strange choice; it’s not one that was made by me, but it is one that I’m stuck with unless the maintainers of the C code are willing to make a modification.

I appreciate your help.