Nested C structs, alignment, and "do not copy fields over"

General Usage
1

I’m writing a Julia interface for a C library. I have two nested C-style structs, where the inner is used by-value. Thes docs say “declare an isbits struct,” but the inner C struct stores pointers, so that isn’t an option. They also say

it is imperative that you do not attempt to manually copy the fields over, as this will not preserve the correct field alignment.

Luckily, I find myself in a situation where the boundaries between the two structs happen to be 8-byte aligned. For example,

mutable struct Inner
   p::Ptr{Cint}
   b::Cchar
end
mutable struct Outer
   a::Inner
   q::Ptr{Cint}
end
mutable struct Combined
   P::Ptr{Cint} # 8 bytes: offset 0
   B::Cchar # 1 byte: offset 8
   Q::Ptr{Cint} # 8 bytes: offset 16
end

The first field of the inner struct (P) and the first field after the inner struct (Q) in Combined happen to be 8-byte aligned. So (apparently) the memory layout matches the corresponding nested C struct. To be safe, I can add guardrails: @assert fieldoffset(Combined, P) % 8 == 0 && field offset(Combined, Q) % 8 == 0. Is there still a good reason why I shouldn’t use Combined?

On the other hand, are there better ways to achieve my aims? I’m making these structs to pass them into @ccall, so I don’t have a lot of flexibility. I must use pointers, and the memory layout must match the C, even if it’s inconvenient.

2

Do you intend to use Inner/Outer in your code on the julia side, or are they purely for FFI? If the latter, you can make Inner immutable and the alignments should match:

julia> struct Inner
           p::Ptr{Cint}
           b::Cchar
       end

julia> isbitstype(Inner)
true

julia> mutable struct Outer # mutable to get a stable pointer for FFI
           a::Inner
           q::Ptr{Cint}
       end

julia> mutable struct Combined
          P::Ptr{Cint} # 8 bytes: offset 0
          B::Cchar # 1 byte: offset 8
          Q::Ptr{Cint} # 8 bytes: offset 16
       end

julia> sizeof(Outer)
24

julia> sizeof(Combined)
24

julia> Base.padding(Outer)
svec(Base.Padding(16, 7))

julia> Base.padding(Combined)
svec(Base.Padding(16, 7))

 # q in Outer
julia> fieldoffset(Outer, 2)
0x0000000000000010

 # q in Combined
julia> fieldoffset(Combined, 3)
0x0000000000000010

# b in Inner
julia> fieldoffset(Inner, 2)
0x0000000000000008

 # B in Combined
julia> fieldoffset(Combined, 2)
0x0000000000000008
3

Do you intend to use Inner /Outer in your code on the julia side, or are they purely for FFI?

Regular usage will involve reading and writing the buffer stored at p. I could maybe have it such that once an Inner object is created, the pointed-to contents of p aren’t modified. But that would be inconvenient and probably bad performance-wise (more allocations).

4

Modifying the contents of the object p points to shouldn’t be a problem at all though, as long as it stays alive for the duration of the FFI call (or as long as C has access to that pointer, if it retains the pointer internally). If Inner is only used for the FFI part and the actual data lives in some julia-controlled allocation, it should work as intended :thinking:

5

I see now: structs containing pointers can be immutable. The pointer must remain unchanged, but the pointed-to data may be overwritten just fine. The confusion stemmed from running isbits(Inner) (false), when what I really wanted was isbitstype(Inner) (true).

Still, it’s inconvenient to be unable to change the pointer. For example, if I want to use cconvert and unsafe_convert to turn something else into an Outer, I ought to allocate x::Outer inside cconvert then set the pointer (x.a.p) inside unsafe_convert. (See the docs here: more detail about my specific use in this post.) But I can’t do that if Inner is immutable.

EDIT: There is a workaround, at the cost of performance: I could allocate a large enough buffer inside cconvert, then copy the data over inside unsafe_convert.

1 Like
6

There’s a subtlety with the wording in the docs here - Ptr{T} objects themselves are isbits just fine, since they are purely defined by their bitpattern (of course, Ptr objects refer to some other memory, but for Julia, the Ptr and the memory it refers to are not the same thing). The case that doesn’t work is when you have some object that isn’t isbits, which ends up being stored as a (transparent) reference in the outer struct. That’s the case for mutable structs or things like arrays or Refs.

Further, the immutable part of a struct only refers to setting the fields of the struct themselves; you can still use the objects stored in those fields as normal (e.g. setting array indices in an array that’s stored in an immutable struct works).

You can still do that, though you have to “replace” the whole instance of Inner in unsafe_convert to “set” the pointer. In cconvert, you can just create the initial Inner with C_NULL as the pointer in order to create Outer. In practice, this is likely going to end up as just storing the pointer instead of all of Inner.

1 Like