Trying to read binary file into struct with julia



i am new to julia. i am trying to read the header of a binary file into a struct. i am using juliapro. i start by defining the struct:

struct CINEheader
    cinetype :: UInt16
    headersize :: UInt16
    compression :: UInt16
    version :: UInt16
    firstMovieImage :: Int32
    totalImageCount :: UInt32
    firstImageNo :: Int32
    imageCount :: UInt32
    offImageHeader :: UInt32
    offSetup :: UInt32
    offImageOffsets :: UInt32

then i open the file and try to read into d

s = open("filename.cine","r")

however when i run this i get a convert error, which i dont undestand

ERROR: MethodError: Cannot `convert` an object of type Type{CINEheader} to an object of type Array{UInt8,1}
This may have arisen from a call to the constructor Array{UInt8,1}(...),
since type constructors fall back to convert methods.

any tips to help me understand what i am doing wrong?


This should work:

s = open("filename.cine","r")
d = Ref{CINEheader}()
read!(s, d)
header = d[]


so i tried your code and now i get:

ERROR: MethodError: no method matching read!(::IOStream, ::Base.RefValue{CINEheader})
Closest candidates are:
  read!(::IO, ::BitArray) at bitarray.jl:2010
  read!(::AbstractString, ::Any) at io.jl:161
  read!(::IO, ::Array{UInt8,N} where N) at io.jl:387


Are you on Julia 0.6? I just tested it on 1.0 and it works on first sight!


Here’s my cine reader, written with dicts instead of structs:

using DataStructures: OrderedDict
using Images: Gray
using FixedPointNumbers, ProgressMeter

function readcine(fname)
    f = open(fname)
    h = OrderedDict()

    # Check magic number
    read(f, UInt16) == UInt(18755) || error(basename(fname), " is not a .cine file")

    fhtypes = OrderedDict(          # File header
        :HeadSize       => UInt16,
        :Compression    => UInt16,
        :Version        => UInt16,
        :FirstImageIndex => Int32,
        :TotalImages    => UInt32,
        :FirstImageNum  => Int32,
        :ImCount        => UInt32,
        :OffImageHeader => UInt32,
        :OffSetup       => UInt32,
        :OffImageOffsetscl => UInt32,
        :TriggerFrac    => UInt32,
        :TriggerSec     => UInt32
    ihtypes = OrderedDict(          # Image header
        :ImHeadSize     => UInt32,
        :Width          => Int32,
        :Height         => Int32,
        :Planes         => UInt16,
        :BitDepth       => UInt16,
        :Comp           => UInt32,
        :SizeImage      => UInt32,
        :PxPerMX        => UInt32,
        :PxPerMY        => UInt32,
        :ClrUsed        => UInt32,
        :ClrImportant   => UInt32,
        :FPS            => UInt16

    for (ID, headertype) in fhtypes     # Read .cine file header
        h[ID] = read(f, headertype)

    seek(f, h[:OffImageHeader])
    for (ID, headertype) in ihtypes     # Read image header
        h[ID] = read(f, headertype)

    bittype = h[:BitDepth] == 8 ? Gray{N0f8} : Gray{N4f12}

    numframes = h[:ImCount]
    numframes > 0 || error("no images exist in file")

    seek(f, h[:OffImageOffsetscl])
    imlocs = read!(f, Array{Int64}(undef, numframes))

    i = 1
    while read(f, UInt16) != 1002   # Get location of images
        i += 1
    h[:ImageOffset] = i

    dt = zeros(numframes)
    skip(f, 2)
    for i = 1:numframes     # Calculate dt for each frame
        fracstart = read(f, UInt32)
        secstart  = read(f, UInt32)
        dt[i] = (secstart - h[:TriggerSec]) + ((fracstart - h[:TriggerFrac])/2^32)
    h[:DeltaT] = dt

    img = zeros(bittype, h[:Height], h[:Width], numframes)
    @showprogress .1 "Loading $(basename(fname)) " for i = 1:numframes
        seek(f, imlocs[i])
        skip(f, read(f, UInt32) - 4)
        img[:,:,i] = rotl90(read!(f, Array{bittype}(undef, h[:Width], h[:Height])))

    return img, h


yes using juliapro which i believe is on 0.6.4?


oh - dont know, but you should look into switching to 1.0 :wink: meanwhile it might work on 0.6 to use an one element array instead of the ref!


will do thanks!


thanks! ill try this out. have you noticed any speed limitations when using dictionaries? im trying to load/manipulate 20 GB phantom videos


The header dictionaries shouldn’t have any real impact on performance–almost all the runtime will be spent in the image-loading loop. You might have trouble fitting 20 GB into memory, though–you may want to load and process one frame at a time if your use case allows it.


I rewrote the reader to allow loading of individual frames:

filename = "test.cine"
vid, header = readcine(filename) # Read the entire video file

# for individual frames:
header = cineheader(filename)         # Just the header
img = readframe(filename, header, 1)  # Read the first frame

# for sets of frames
subvid1 = readframe(filename, header, 1:10:500) # Read every tenth frame
subvid2 = readframe(filename, header, [1, 7, 42]) # Read arbitrary sets of frames