You should absolutely parameterize the struct, unless you only need it for UInt8, since Vector{<:Integer} is an abstract type, which will cause allocations.
Also, the struct should not be mutable, if you only modify the member fields and not the struct itself. Just remove mutable.