Reinterpret vector of mixed-type tuples

General Usage
1

I want to reinterpret a vector of mixed-type tuples as a vector of UInt8, however, I receive an error:

b = reinterpret(UInt8, Tuple{UInt8, UInt32}[(1, 2)]);
b[1]

This is very close to the docs example of reinterpret, where

b = reinterpret(UInt32, Tuple{UInt8, UInt32}[(1, 2)]);
b[1]

is expected to fail because the types don’t align.
However, everything should align with UInt8.
Also strange, that

b = reinterpret(UInt8, Tuple{UInt32, UInt32}[(1, 2)]);
b[1]

succeeds.
I suspect, it has to do with the fact that

julia> Base.array_subpadding(UInt8, Tuple{Int, Bool})
false

which is quite surprising.

1 Like
2

Interestingly, writing is allowed …

julia> a = [(0x1, 0x00002)]
1-element Vector{Tuple{UInt8, UInt32}}:
 (0x01, 0x00000002)

julia> b = reinterpret(UInt8, a);

julia> b[1] = 10
10

julia> a
1-element Vector{Tuple{UInt8, UInt32}}:
 (0x0a, 0x00000002)

Could it be that readable and writable are exchanged in julia/base/reinterpretarray.jl at master · JuliaLang/julia · GitHub?
At least, I am puzzled to see that I can edit a reinterpreted array, but not read it.

3

The issue is struct padding and/or array padding. Despite the fact that a Tuple{UInt8, UInt32} could fit in 5 btyes, it actually uses 8 in most situations.

julia> sizeof(Tuple{UInt8, UInt32})
8

julia> fieldoffset(Tuple{UInt8, UInt32}, 1)
0x0000000000000000

julia> fieldoffset(Tuple{UInt8, UInt32}, 2)
0x0000000000000004

So the layout is

[ UInt8  UNUSED  UNUSED  UNUSED  UInt32__UInt32__UInt32__UInt32 ]

My recommendation is to use a Tuple{UInt32, UInt32} instead. At the end of the day it has the same layout because of the padding.

julia> reinterpret(UInt8, Tuple{UInt32, UInt32}[(1, 2)])
8-element reinterpret(UInt8, ::Vector{Tuple{UInt32, UInt32}}):
 0x01
 0x00
 0x00
 0x00
 0x02
 0x00
 0x00
 0x00

You can

julia> reinterpret(NTuple{5, UInt8}, (0x1, 0x00002))
(0x01, 0x02, 0x00, 0x00, 0x00)

but I can’t explain exactly why this uses a different layout. Probably because it never actually uses memory, it’s just two values that are in registers anyway so it just mashes them together.

Knowing what we know about padding, we get a little closer with

julia> reinterpret(NTuple{8,UInt8}, Tuple{UInt8, UInt32}[(1, 2)])
1-element reinterpret(NTuple{8, UInt8}, ::Vector{Tuple{UInt8, UInt32}}):
Error showing value of type Base.ReinterpretArray{NTuple{8, UInt8}, 1, Tuple{UInt8, UInt32}, Vector{Tuple{UInt8, UInt32}}, false}:
ERROR: Padding of type NTuple{8, UInt8} is not compatible with type Tuple{UInt8, UInt32}.

which does manage to make the array, but accessing it fails (hence Error showing value).

There are more nefarious ways to make this work BUT YOU REALLY SHOULD NOT DO THEM BECAUSE THEY CAN CAUSE WILD AND CRAZY BUGS.

(they look something like this, but I'm no expert and this is a BAD IDEA(tm) anyway) 
julia> X = [(0x1, 0x00002)]
1-element Vector{Tuple{UInt8, UInt32}}:
 (0x01, 0x00000002)

julia> unsafe_wrap(Vector{UInt8}, Ptr{UInt8}(pointer(X)), sizeof(X))
8-element Vector{UInt8}:
 0x01
 0x5b # garbage
 0xcd # garbage
 0x98 # garbage
 0x02
 0x00
 0x00
 0x00
3 Likes
4

The problem is that for that reason BSON fails to write Arrays of mixed-type tuples.
Hence, I cannot store some molgraphs from MolecularGraph.
But thanks for pointing to the right location!

5

Had the time to read this with a computer at hand. Really, very helpful insights.