Unfortunately I still don’t understand what you want to do here.
Thanks for pointing this out. I constantly struggle with either writing a whole novel which seems to lead people to not want to read it or failing to produce enough context for the problem to be understandable.
What I am doing is decoding/parsing unstructured binary files with an ad-hoc protocol, trying to make things both very fast (goal is to take < 1 second to decode a 1 GB file) while allowing for users to hook into the machinery to e.g deal with data too large to fit in memory.
For the latter, I allow users to inspect the current decoded state which is just a Vector{UInt8} with some surrounding metadata (types basically) on how to interpret it. A chunk of data has a header which tells the size of the next chunk and points to a line in a text-spec saying what types of data are in the chunk (e.g the next following 22 bytes is an UInt16 followed by a Float64 followed by a Int32 etc…). First step of decoding is basically appending the chunks of each unique line in the spec into a single Vector{UInt8} for that line in the spec.
viewrows is basically used in conjunction with the type metadata to select a slice of the raw data blob and reinterpret it to the right type so that that strided steprange of UInt8 gets reinterpreted into e.g. a vector of the N:th type in that chunktype. And no, I did not invent this protocol and I don’t have the means to change it.
In the current state, it is a bit of a thorn in the side that should a user make use of the actual values (and not the metadata) to determine if they want a piece of data, the program will fail because after you have viewed the values its no longer possible to append more chunks due to the reshape issue.
Maybe you can reformulate the problem so that you can use the excellent ElasticArrays.jl
This looks like it could be very useful as I could then store the data in matrix shape and append to it. Then there would be no need to reshape the data and therefore the seal is broken. Thank you very much! I will mark your post as the solution once I have verified that it works.