Disable memory alignment inside struct

General Usage
,
1

I have a (somewhat old) file that was created from a Fortran script and contains a struct without any padding. I try to read the file as follows

struct Foo
    id::UInt32
    Value::Int64
end
io = open(filepath, "r")
rows = Mmap.mmap(io, Vector{Foo}, nrows)
sa = StructArray(rows)

However, because the default behavior is to add padding between id and Value the size of Foo is wrong and I get nonsense.

Is there a way to disable the padding?

My main concern here is to optimize loading the file into a StructArray, as the files are ~10GB and I need to load hundreds of them in my computations while also query the entire files. If there is a better way to load them Iā€™d be happy to learn.

2

Maybe FortranFiles.jl is what you need?

But unless you are reading these only once, maybe it is worthwhile to convert them to Arrow or other modern format.

1 Like