Unexpected struct size on aarch64 (macOS M)

General Usage
1

(see also Startup issue on recent macOS on M3 · Issue #29 · felipenoris/Oracle.jl · GitHub)

We’d like to use Oracle.jl for a project and the initial port went fine, until someone wanted to run things on their macbook. There the Oracle driver bails out because a structure does not match its C-side equivalent.

In a nutshell, Oracle.jl defines C functions that essentially return sizeofof each relevant struct, and on loading the library, it will start by verifying that these functions return the same as Julia-side sizeof on the equivalent structs. Works swell on x64.

On M-series macs, however, these definitions give an issue:

# 40 bytes -> sizeof dpiDataBuffer on 64bit arch
primitive type OraDataBuffer 40 * 8 end

struct OraData
    is_null::Int32 # Specifies if the value refers to a null value (1) or not (0).
    value::OraDataBuffer
end

the equivalent C structure in the Oracle library that Oracle.jl uses has a C definition that is equivalent:

// structure used for transferring data to/from ODPI-C
struct dpiData {
    int isNull;
    dpiDataBuffer value;
};

but where dpiDataBuffer is a union of a whole list of things.

In any case, both on x64 and aarch64, the size of this C struct is 48 bytes - 4 bytes for the isNull, alignment bytes, then 40 bytes for the dpiDataBuffer. So while the primitive type definition above is a bit of a hack/shortcut, the 40 bytes looks correct here.

On x64, sizeof(OraDataBuffer) returns the same value, 48. On macOS, however, it gets - apparently - rounded up to 64 bytes. I have no clue why, the closest I’ve come to grokking this is the definition for MAX_ALIGN in julia_internals.h which sets 4 bytes on Intel, 16 bytes on Apple M series, and that matches what I seem to be observing somewhat.

Anyway, sorry for all the context, my question is: how can I convince Julia that this struct is, in fact, just 48 bytes? Or is the only solution that I need to somehow figure out how to make the struct larger/“more aligned” C-side? That might be doable using a vendored version of the Oracle library, but… well, needless to say, not really a preferred direction :slight_smile:

(the version of macOS I tested on has clang 16 and I used multiple recent versions of Julia 1.11 to exclude our choice of using a Nix-managed version of Julia)

2

First of all, I get

julia> # 40 bytes -> sizeof dpiDataBuffer on 64bit arch
       primitive type OraDataBuffer 40 * 8 end

julia> struct OraData
           is_null::Int32 # Specifies if the value refers to a null value (1) or not (0).
           value::OraDataBuffer
       end

julia> sizeof(OraDataBuffer)
40

julia> sizeof(OraData)
64

on both Linux and macOS, both x86_64 and aarch64, so I don’t think this is specific to macOS.

Then, I believe the thing is is that for a primitive type the alignment is at 16 bytes:

julia> Base.aligned_sizeof(OraDataBuffer)
48

and so the actual size of the object in a field of a struct is 48 bytes, not 40.

But your OraDataBuffer is actual struct, not a primitive type. If you create an actual struct of size 40, like (I’m making the layout up)

julia> struct OraDataBufferStruct
           a::Int64
           b::Int64
           c::Int64
           d::Int64
           e::Int64
       end

julia> sizeof(OraDataBufferStruct)
40

julia> Base.aligned_sizeof(OraDataBufferStruct)
40

and

julia> struct OraData
           is_null::Int32
           value::OraDataBufferStruct
       end

julia> sizeof(OraData)
48

julia> Base.aligned_sizeof(OraData)
48

Edit: sorry, I read it wrong, you said correctly that dpiDataBuffer is a union, but it probably takes the size of 40 from one of the structs part of the union.

3

Thanks, that’s helpful (I don’t have Linux on aarch64 nor mac on x64, so could not check whether it’s processor-specific).

I guess I’ll need to go through the Oracle.jl code (I’m not the author, in case that that wasn’t obvious ;-)), and see how value is used. My hunch is that it’s just there as a placeholder, so morphing the definition of OraDataBuffer into a struct with filler fields should actually work out fine.

4

This is very helpful @giordano!

One thing I would add is that this size check always existed in the package, since 2019, so Julia v1.0 I guess. I added this check because I’m defining Julia structs that are passed to the C library, which also defines the very same structs, and I would like to check struct changes when updating the underlying C library to new versions.

If you check the latest commit CI results, update gitignore · felipenoris/Oracle.jl@129fb75 · GitHub, you’ll see that this check passes all Julia versions up to v1.11. But it fails on nightly on both Linux and macOS (no the CI does not use ARM processos as far as I know).

So I guess you’re running Julia nightly on your results?

5

Not on aarch64-darwin (or not only, I get the same in v1.11 and nightly), but yes for x86_64-linux, which may explain the difference: the alignment of primitive types changed in LLVM: [CI] Failing `datatype` tests on x86 Linux · Issue #853 · JuliaParallel/MPI.jl · GitHub

Well, it does:

macOS-latest is an aarch64-darwin system. But you’re forcing to use the the x86_64 binaries: