Looking at some more background information based on the previous comment, I searched for append to a file. I came across this topic which is most useful. This led me to the section in the Arrow manual that describes the writer function. An example is
julia> writer = open(Arrow.Writer, "test.arrow");
julia> for j = 1:400:100_000
A = Tables.table(transpose(reshape(1:400, 4, 100)) .+ j)
Arrow.write(writer, A)
end
julia> close(writer)
julia> table = Arrow.Table("test.arrow")
Arrow.Table with 24900 rows, 4 columns, and schema:
:Column1 Int64
:Column2 Int64
:Column3 Int64
:Column4 Int64
julia> table[1][1:3]
3-element Vector{Int64}:
2
6
10
This successfully writes out data from 4 channels sampled at 51.2 kHz in 1 second chunks. That works in both Float32 and Float64 using the SD card on PI for a rate of 16 Mbits/s. Sometimes on the first invocation it gives a buffer overrun error. I assume this is due to JIT compilation.
Now my wish is to also save some structures to the same file to contain the test and channel metadata. I see that there is an Arrow.struct but no example on how to save it, let alone save it in conjunction with my table data.
Is this possible?