Reinterpret to existing vector

Lets say I have a byte (UInt8) array that looks like this:

source = reinterpret(UInt8, Int64[10,11])
 0x0a
 0x00
 0x00
 0x00
....

And an existing allocated array of two integers
target = zeros(Int64, 2)

I basically want to say reinterpret!(target, source) where the UInt8 array is reinterpreted as an Int64 to target (So I don’t allocate new vectors). I thought of using unsafe_store or unsafe_copyto! but didn’t figure it out yet.

What would be the way to go? :smiley:

I am no expert on this but if you reinterpret your target you should not allocate anything.
So this little snippet should work.

a = reinterpret(UInt8, Int16[10,23])
target = zeros(Int16, 2)
b = reinterpret(UInt8, target)
unsafe_copyto!(pointer(b), pointer(a), min(length(b), length(a)))
println(target)
1 Like

The line
b = reinterpret(UInt8, target)
Will allocate

@btime reinterpret(UInt8, target)
92.453 ns (1 allocation: 32 bytes)

This, for example, doesn’t allocate a new array:

x = Int64[1 2 3 4 ; 5 6 7 8]
s = reinterpret(UInt8, @view x[2:2,3:3] )
s[1] = 0x0a
x

The assignment in the 3rd line modifies the original array.

Interesting, maybe it allocates the structure Base.ReinterpretArray. The original array, however, is changed by assigning to the reinterpreted array so there should be no reallocation of it.

julia> @time a = zeros(UInt8,20);
  0.000001 seconds (1 allocation: 80 bytes)

julia> @time b = reinterpret(UInt16,a);
  0.000003 seconds (1 allocation: 32 bytes)

julia> @time a = zeros(UInt8,100);
  0.000001 seconds (1 allocation: 160 bytes)

julia> @time b = reinterpret(UInt16,a);
  0.000003 seconds (1 allocation: 32 bytes)

1 Like

This does allocate,

@btime reinterpret(UInt8, @view x[2:2,3:3] )
115.531 ns (2 allocations: 144 bytes)

Wasn’t really properly doing my benchmark here :sweat_smile: (see below)

It doesn’t allocate the array. Check this:

x = collect( 1:2^30 )
@time s = reinterpret(UInt8, @view x[17:2^29] )
length(s)

After running this, s is a gigantic array – but only a handfull allocations have been performed.

1 Like

I know it doesn’t allocate the array itself, but it does allocate. Since the memory for the output array (target) is already allocated it should be possible to move the bytes there without any allocations. Similar to just copying values from one array to another (which doesn’t allocate at all)

(Also this doesn’t answer the question to interpret a byte array to int, not the other way around)

If you put it in a function I think it does not allocate anymore as it gets optimized.

julia> function test!(a,b)
       a_uint8 = reinterpret(UInt8, a)
       b_uint8 = reinterpret(UInt8, b)
       unsafe_copyto!(pointer(b_uint8), pointer(a_uint8), min(length(b_uint8), length(a_uint8)))
       nothing
end

julia> a = rand(UInt8,4)
4-element Vector{UInt8}:
 0x6f
 0x9b
 0x5b
 0x64

julia> b = zeros(UInt16,2)
2-element Vector{UInt16}:
 0x0000
 0x0000

julia> @timev test!(a,b)
  0.000001 seconds
elapsed time (ns):  1380
gc time (ns):       0
bytes allocated:    0
pool allocs:        0
non-pool GC allocs: 0
minor collections:  0
full collections:   0

julia> b
2-element Vector{UInt16}:
 0x9b6f
 0x645b

4 Likes

You are right, as memcpy also doesn’t allocate that way

using BenchmarkTools
function test()
    source = reinterpret(UInt8, Int64[10,11])
    target = zeros(Int64, 2)
    @btime ccall(:memcpy, Ptr{Cvoid}, (Ptr{Cvoid}, Ptr{Cvoid}, Csize_t), $target, $source, length($source))
end
test()

Or

@ccall memcpy(pointer(target)::Ptr{Cvoid}, pointer(source)::Ptr{Cvoid}, length(source)::Cssize_t)::Cvoid

A bit neater

This is just an artefact of your benchmarking code. It does not allocate. Benchmark like this

@btime b = reinterpret(UInt8, $target)
2 Likes

Yeah you are right, realized that too :sweat_smile:, the answer then is actually simply:
b = reinterpret(Int64, source) where I don’t have to fill target at all

Would such a thing be acceptable?

target .= source.parent

or

target = source.parent
1 Like

Looking at this again, this only works after the reinterpret?

Yes. Is not that your case?

I just used the reinterpret to create the example vector, the actual source array are just bytes from a file