I think IO and Array are interesting non-orthogonal APIs. This is sort of the same idea as using mmap when the file is known to be constant size and fit in memory. The IOBuffer constructor is similar to that, and can help overlay complex zero-copy formats on any byte-indexed array:
julia> typeof(IOBuffer(view(UInt8[], :)))
Base.GenericIOBuffer{SubArray{UInt8,1,Array{UInt8,1},Tuple{Base.Slice{Base.OneTo{Int64}}},true}}
But then, unstructured formats are not enjoyable to work with, so you might wrap this again in an access-api. But then, that could just contain another flat byte buffer, and keep nesting further.