# Dealing with complex C structure
**URL:** https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045
**Category:** New to Julia
**Tags:** ccall
**Created:** [February 14, 2018, 2:06am UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045 "2018-02-14T02:06:53Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [February 14, 2018, 2:06am UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/1 "2018-02-14T02:06:53Z")
The [official Julia doc for calling C/Fortran code](https://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/) says:
> Packed structs and union declarations are not supported by Julia.
So, does that mean I won’t be able to call the C code? I’m trying to deal with code that takes the following structure… thanks for any kind help here.
```julia
struct abc
{
union {
double x;
char *y;
struct {
struct abc *array;
unsigned short a;
unsigned short b;
} z;
} w;
unsigned short v;
}
```
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [February 14, 2018, 5:28am UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/2 "2018-02-14T05:28:57Z")
Taking the approach from the doc:
> You can get a near approximation of a union if you know, a priori, the field that will have the greatest size (potentially including padding). When translating your fields to Julia, declare the Julia field to be only of that type.
That’s not too bad. Here’s a MWE:
```C
/* test.c */
typedef struct abc
{
union {
unsigned int a;
long b;
} ab;
int c;
} ABC, *ABCP;
ABC make1()
{
ABC x;
x.c = 1;
x.ab.b = 2;
return x;
}
ABC make2()
{
ABC x;
x.c = 1;
x.ab.a = 2;
return x;
}
```
Compile as usual:
```julia
$ gcc -fPIC -shared test.c -o libtest.so
```
In Julia, I can treat the union as a regular `long` variable (which is the largest in this example):
```julia
julia> const TEST = "/tmp/libtest.so"
"/tmp/libtest.so"
julia> x = ccall((:make1, TEST), ABC, ())
ABC(2, 1)
julia> y = ccall((:make2, TEST), ABC, ())
ABC(5815918277548834818, 1)
```
So the value returned by `make2` is obviously messed up because I should be looking for the `int` rather than the `long` field. To correct that, I can do some bit twiddling (not my favorite but some people here like it 🙂 )
```julia
julia> UInt64(y.ab)
0x50b64adc00000002
julia> Int32(y.ab & 0x00000000ffffffff)
2
```
I really need a way to reinterpret the thing so perhaps I just go with a fixed size tuple.
```julia
julia> struct ABC2
ab::NTuple{8, Cuchar} # 8 bytes
c::Int32
end
julia> z = ccall((:make2, TEST), ABC2, ())
ABC2((0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00), 1)
julia> reinterpret(Int32, collect(z.ab[1:4]))
1-element Array{Int32,1}:
2
```
Next, I’ll try to reinterpret to another struct. Hopefully that will work.
Does this all make sense? Is there any better way to approach this?
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [February 14, 2018, 12:20pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/3 "2018-02-14T12:20:49Z")
An easy “implementation-defined” variant on 64 bit is the following:
```julia
struct abc_x
a::Float64
padding::Int32
v::UInt16
end
struct abc_y
y::Ptr{UInt8}
padding::Int32
v::UInt16
end
struct abc_z
array::Ptr{Void}
a::UInt16
b::UInt16
v::UInt16
end
@assert sizeof(abc_x)==sizeof(abc_y)==sizeof(abc_z);
```
You can freely reinterpret between these three variants and pass any of them to your C function (and reinterpret outputs). If you want to be fancy then you can give the pointer `abc` a more reasonable type.
You get no UB with respect to the explicit padding bytes (if the padding was implicit in julia, then reinterpret can give undefined results; basically the reinterpret vomits register content into the padding, just like C; but C knows that all maybe-relevant bytes in the union must be preserved)
If your C code has `#pragma pack` then you need to figure out the layout, by debugger or spec-reading (in this specific example there should be no difference). I am not entirely sure how to make a variant that also works on 32bit; maybe use an `if sizeof(Ptr{Int})==8` and write two versions by hand? Do you care about 32bit?
(advantage of using int/ptr-types instead of `NTuple{8,UInt8}`: endian-ness is automatically correct; but everything is little-endian today anyway, so whatever. Even if the other side expects network byte order, the int-type tells julia about the alignment)
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [February 14, 2018, 2:17pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/4 "2018-02-14T14:17:55Z")
Post-script: I think the canonical 0.7 variant would be something like
```julia
struct abc
f1::UInt64
f2::UInt16
f3::UInt16
f4::UInt16
end
```
with `getproperty` and `setproperty!(Ptr{abc}, ...)` and constructors overloaded (so no user-facing reinterpret needed, and the nesting is the same as in C).
Someone should write a package for this (which preferably also does `#pragma pack`, endian-ness and 32bit), with a macro `@julia_struct_from_C_like` that does all the computations and definitions (maybe even in a way that can eat the header-file post pre-processor). Maybe Keno’s `Cxx.jl` already does the job? Maybe it will do the job in the future?
I am not entirely whether this always works, though; that is, whether julia gets confused when writing into an `UInt16` that is part of a `UInt64` (they are a union; but aliasing / alignment analysis may get confused in the end, and you want the `UInt16`-write to be atomic and 2-byte-aligned, and not modify the remaining bits in multi-threaded code; on the other hand, you want the `UInt64` reads/writes to be 8-byte-aligned, so reinterpreting from four `UInt16` does not cut the cake; what is clang doing?)
(personally, I would also make the meaningless structure padding at the end explicit; you save some cycles by letting them undefined, but then you avoid all headaches of “did I handle a password/crypto and need to sanitize paddings before serialization?” and avoid all possible human errors in size-calculations; so just add another `padding_end::UInt16` at the end)
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [February 14, 2018, 3:18pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/5 "2018-02-14T15:18:50Z")
I don’t have time to dig more at the moment but a quick test shows that reinterpreting `Ptr{Void}` as a `Float64` crashes julia. Will come back to this later tonight.
```julia
julia> struct DEF2
mess1::Ptr{Void}
a::UInt16
b::UInt16
pad2::NTuple{4, Cuchar}
v::Int16
pad3::NTuple{6, Cuchar}
end
julia> z = ccall((:make5, TEST), DEF2, ())
DEF2(Ptr{Void} @0x4008000000000000, 0x19e8, 0x0c71, (0x9f, 0x7f, 0x00, 0x00), 1, (0x00, 0x00, 0x00, 0x00, 0x00, 0x00))
julia> reinterpret(Float64, z.mess1)
LLVM ERROR: Cannot select: 0x7f9f0aa2b520: f64 = truncate 0x7f9f0b89cfc0
0x7f9f0b89cfc0: i64,ch = CopyFromReg 0x7f9f09c5c030, Register:i64 %vreg1
0x7f9f0a9df690: i64 = Register %vreg1
In function: julia_reinterpret_62748
```
where the C code looks like:
```julia
typedef struct def
{
union {
double x; /* 8 bytes */
char *y; /* 8 bytes as for 64-bit system */
struct {
struct def *array; /* 8 bytes */
unsigned short a; /* 2 bytes */
unsigned short b; /* 2 bytes + padded 4 bytes */
} z;
} w;
unsigned short v; /* 2 bytes -> padded to 8 bytes (64-bit system) */
} DEF, *DEFP;
DEF make5()
{
DEF x;
x.v = 1;
x.w.x = 3.0;
return x;
}
```
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [February 14, 2018, 3:39pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/6 "2018-02-14T15:39:38Z")
That’s what I get for mixing julia versions. Reinterpret between Float64 and pointers works fine on 0.7; on 0.6 you can go via `reinterpret(Float64, reinterpret(UInt64,C_NULL))`.
So you could put a reinterpret to a pure uint-struct in beetween? This should be a nop, but is really ugly.
I guess this is a bug. I’ll see whether I find it in the tracker and otherwise open an issue.
Also, are you sure about your offsets? I thought the C struct would be:
```julia
typedef struct def
{ /* always 8-byte aligned; has bytes 1:16 */
union {
double x; /* bytes 1:8 */
char *y; /* bytes 1:8 */
struct {
struct def *array; /* bytes 1:8 */
unsigned short a; /* bytes 9:10*/
unsigned short b; /* bytes 11:12 */
} z; /* bytes 1:12 */
} w; /* bytes 1:12 */
unsigned short v; /* bytes 13:14*/
/*Padding: bytes 15:16 ; filled with UB*/
} DEF, *DEFP;
```
If you get away with passing pointers to structs, then all this conversion hell goes away: build your chunk of memory somewhere and pass the pointer to the ccall. Then you would just reinterpret the pointer and unsafe\_load/store! in order to modify/reinterpret your union.
---
### Author:  [@tk3369](https://discourse.julialang.org/u/tk3369)
#### Post date: [February 14, 2018, 5:48pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/7 "2018-02-14T17:48:41Z")
Looks like UInt64 is most _reinterpretation-friendly_ for 8-byte fields. I can reinterpret it to double or char\* or a pointer. The only drawback is that if I have anything smaller than 8 bytes then I would have to shift/convert.
> [@foobar\_lv2](#):
>
> So you could put a reinterpret to a pure uint-struct in beetween?
Sorry, I don’t understand this question.
> [@foobar\_lv2](#):
>
> Also, are you sure about your offsets?
Yes, I’m certain as I went through a couple of scenarios. Apparently gcc requires the inner struct to be 8-byte aligned as well so it added an extra 4 bytes for that.
---
### Author:  [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)
#### Post date: [February 14, 2018, 6:20pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/8 "2018-02-14T18:20:45Z")
So I opened an issue on [https://github.com/JuliaLang/julia/issues/26053](https://github.com/JuliaLang/julia/issues/26053).
A quick fix for your code is
```julia
Base.reinterpret(::Type{Ptr{T}}, x::Float64) where T = reinterpret(Ptr{T}, reinterpret(UInt64, x))
Base.reinterpret(::Type{Float64}, x::Ptr) = reinterpret(Float64, reinterpret(UInt64, x))
```
Unfortunately I don’t understand the code-base well enough to submit this as a PR; someone who really knows it needs to check whether there are other pointer-reinterpret cases that make problems.
(I tested the `@code_llvm` and correctness for this quick-fix, but have no idea whether there are other problems; also, you need to modify this for 32 bit systems )
(also this might not emit the fastest `@code_native` because of register-juggling between xmm and general purpose registers; but this shouldn’t matter for your case)
---
### Author:  [@ihnorton](https://discourse.julialang.org/u/ihnorton)
#### Post date: [February 17, 2018, 3:10pm UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/9 "2018-02-17T15:10:06Z")
See also StrPack.jl and StructIO.jl. They don’t know about unions as far as I can tell, but IIRC at least StrPack knows about packing rules. Perhaps getters/setters could be created automatically with getproperty overloading. I believe StructIO is the more modern/updated because keno and staticfloat are using it for object file work, so ideally add or port any missing functionality there.
---
### Author:  [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [February 19, 2018, 9:49am UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/10 "2018-02-19T09:49:18Z")
I’m trying to wrap SDL2 Events but I’m not sure I’m doing it right.
SDL Event is a big union of all the possible [event types](https://github.com/spurious/SDL-mirror/blob/master/include/SDL_events.h#L525). But they make sure it’s 52 bytes long, so I think Clang.jl did the right thing here:
```julia
mutable struct Event
_Event::NTuple{56, Uint8}
end
```
I can declare and fill an Event with:
```julia
ev = SDL2.Event(ntuple(i->-1,56))
SDL2.PollEvent(pointer_from_objref(ev))
```
Then the first field of this indicates which type of event it is (in this particular case 1024 is a `WindowEvent`), so I can try to convert it:
```julia
unsafe_load( Ptr{SDL2.WindowEvent}(pointer_from_objref(ev)) )
SDL2.WindowEvent(1024, 2856, 1, 0, 0, 495, 544, 10, 10)
```
This seems to work but I’m not sure if it’s correct and it’s a bit ugly, is there a better way to do it ?
---
### Author:  [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [February 27, 2018, 4:54am UTC](https://discourse.julialang.org/t/dealing-with-complex-c-structure/9045/11 "2018-02-27T04:54:02Z")
I had to solve a similar issue with Gtk many years ago: [https://github.com/JuliaGraphics/Gtk.jl/blob/d0a218011bbb3e30934bdabd780131d1eaa6e3d0/src/gdk.jl#L74](https://github.com/JuliaGraphics/Gtk.jl/blob/d0a218011bbb3e30934bdabd780131d1eaa6e3d0/src/gdk.jl#L74)
Although these days, I might instead use `getproperty` overloading to simulate union-field access on the `mutable struct Event` object – basically what you are doing, and wrapped in a nice API so that users don’t see the ugly `@gc_preserve` / `pointer_from_objref` / `unsafe_load` mess