Trying to read binary file into struct with julia

New to Julia
,
1

i am new to julia. i am trying to read the header of a binary file into a struct. i am using juliapro. i start by defining the struct:

struct CINEheader
    cinetype :: UInt16
    headersize :: UInt16
    compression :: UInt16
    version :: UInt16
    firstMovieImage :: Int32
    totalImageCount :: UInt32
    firstImageNo :: Int32
    imageCount :: UInt32
    offImageHeader :: UInt32
    offSetup :: UInt32
    offImageOffsets :: UInt32
 
end

then i open the file and try to read into d

s = open("filename.cine","r")
d=CINEheader
read(s,d)

however when i run this i get a convert error, which i dont undestand

ERROR: MethodError: Cannot `convert` an object of type Type{CINEheader} to an object of type Array{UInt8,1}
This may have arisen from a call to the constructor Array{UInt8,1}(...),
since type constructors fall back to convert methods.

any tips to help me understand what i am doing wrong?

2

This should work:

s = open("filename.cine","r")
d = Ref{CINEheader}()
read!(s, d)
header = d[]
3

so i tried your code and now i get:

ERROR: MethodError: no method matching read!(::IOStream, ::Base.RefValue{CINEheader})
Closest candidates are:
  read!(::IO, ::BitArray) at bitarray.jl:2010
  read!(::AbstractString, ::Any) at io.jl:161
  read!(::IO, ::Array{UInt8,N} where N) at io.jl:387
4

Are you on Julia 0.6? I just tested it on 1.0 and it works on first sight!

5

Here’s my cine reader, written with dicts instead of structs:

using DataStructures: OrderedDict
using Images: Gray
using FixedPointNumbers, ProgressMeter

function readcine(fname)
    f = open(fname)
    h = OrderedDict()

    # Check magic number
    read(f, UInt16) == UInt(18755) || error(basename(fname), " is not a .cine file")

    fhtypes = OrderedDict(          # File header
        :HeadSize       => UInt16,
        :Compression    => UInt16,
        :Version        => UInt16,
        :FirstImageIndex => Int32,
        :TotalImages    => UInt32,
        :FirstImageNum  => Int32,
        :ImCount        => UInt32,
        :OffImageHeader => UInt32,
        :OffSetup       => UInt32,
        :OffImageOffsetscl => UInt32,
        :TriggerFrac    => UInt32,
        :TriggerSec     => UInt32
    )
    ihtypes = OrderedDict(          # Image header
        :ImHeadSize     => UInt32,
        :Width          => Int32,
        :Height         => Int32,
        :Planes         => UInt16,
        :BitDepth       => UInt16,
        :Comp           => UInt32,
        :SizeImage      => UInt32,
        :PxPerMX        => UInt32,
        :PxPerMY        => UInt32,
        :ClrUsed        => UInt32,
        :ClrImportant   => UInt32,
        :FPS            => UInt16
    )

    for (ID, headertype) in fhtypes     # Read .cine file header
        h[ID] = read(f, headertype)
    end

    seek(f, h[:OffImageHeader])
    for (ID, headertype) in ihtypes     # Read image header
        h[ID] = read(f, headertype)
    end

    bittype = h[:BitDepth] == 8 ? Gray{N0f8} : Gray{N4f12}

    numframes = h[:ImCount]
    numframes > 0 || error("no images exist in file")

    seek(f, h[:OffImageOffsetscl])
    imlocs = read!(f, Array{Int64}(undef, numframes))

    seekstart(f)
    i = 1
    while read(f, UInt16) != 1002   # Get location of images
        i += 1
    end
    h[:ImageOffset] = i

    dt = zeros(numframes)
    skip(f, 2)
    for i = 1:numframes     # Calculate dt for each frame
        fracstart = read(f, UInt32)
        secstart  = read(f, UInt32)
        dt[i] = (secstart - h[:TriggerSec]) + ((fracstart - h[:TriggerFrac])/2^32)
    end
    h[:DeltaT] = dt

    img = zeros(bittype, h[:Height], h[:Width], numframes)
    @showprogress .1 "Loading $(basename(fname)) " for i = 1:numframes
        seek(f, imlocs[i])
        skip(f, read(f, UInt32) - 4)
        img[:,:,i] = rotl90(read!(f, Array{bittype}(undef, h[:Width], h[:Height])))
    end

    close(f)
    return img, h
end
6

yes using juliapro which i believe is on 0.6.4?

7

oh - dont know, but you should look into switching to 1.0 :wink: meanwhile it might work on 0.6 to use an one element array instead of the ref!

8

will do thanks!

9

thanks! ill try this out. have you noticed any speed limitations when using dictionaries? im trying to load/manipulate 20 GB phantom videos

10

The header dictionaries shouldn’t have any real impact on performance–almost all the runtime will be spent in the image-loading loop. You might have trouble fitting 20 GB into memory, though–you may want to load and process one frame at a time if your use case allows it.

11

I rewrote the reader to allow loading of individual frames: https://gist.github.com/stillyslalom/36d72c216da18202d018025d3ef28fff

filename = "test.cine"
vid, header = readcine(filename) # Read the entire video file

# for individual frames:
header = cineheader(filename)         # Just the header
img = readframe(filename, header, 1)  # Read the first frame

# for sets of frames
subvid1 = readframe(filename, header, 1:10:500) # Read every tenth frame
subvid2 = readframe(filename, header, [1, 7, 42]) # Read arbitrary sets of frames