Lets say I have a byte (UInt8
) array that looks like this:
source = reinterpret(UInt8, Int64[10,11])
0x0a
0x00
0x00
0x00
....
And an existing allocated array of two integers
target = zeros(Int64, 2)
I basically want to say reinterpret!(target, source)
where the UInt8
array is reinterpreted as an Int64
to target
(So I don’t allocate new vectors). I thought of using unsafe_store
or unsafe_copyto!
but didn’t figure it out yet.
What would be the way to go?
I am no expert on this but if you reinterpret your target you should not allocate anything.
So this little snippet should work.
a = reinterpret(UInt8, Int16[10,23])
target = zeros(Int16, 2)
b = reinterpret(UInt8, target)
unsafe_copyto!(pointer(b), pointer(a), min(length(b), length(a)))
println(target)
1 Like
The line
b = reinterpret(UInt8, target)
Will allocate
@btime reinterpret(UInt8, target)
92.453 ns (1 allocation: 32 bytes)
DOT
January 28, 2023, 7:22pm
4
This, for example, doesn’t allocate a new array:
x = Int64[1 2 3 4 ; 5 6 7 8]
s = reinterpret(UInt8, @view x[2:2,3:3] )
s[1] = 0x0a
x
The assignment in the 3rd line modifies the original array.
Interesting, maybe it allocates the structure Base.ReinterpretArray
. The original array, however, is changed by assigning to the reinterpreted array so there should be no reallocation of it.
julia> @time a = zeros(UInt8,20);
0.000001 seconds (1 allocation: 80 bytes)
julia> @time b = reinterpret(UInt16,a);
0.000003 seconds (1 allocation: 32 bytes)
julia> @time a = zeros(UInt8,100);
0.000001 seconds (1 allocation: 160 bytes)
julia> @time b = reinterpret(UInt16,a);
0.000003 seconds (1 allocation: 32 bytes)
1 Like
This does allocate,
@btime reinterpret(UInt8, @view x[2:2,3:3] )
115.531 ns (2 allocations: 144 bytes)
Wasn’t really properly doing my benchmark here (see below)
DOT
January 28, 2023, 7:31pm
7
It doesn’t allocate the array. Check this:
x = collect( 1:2^30 )
@time s = reinterpret(UInt8, @view x[17:2^29] )
length(s)
After running this, s
is a gigantic array – but only a handfull allocations have been performed.
1 Like
I know it doesn’t allocate the array itself, but it does allocate. Since the memory for the output array (target
) is already allocated it should be possible to move the bytes there without any allocations. Similar to just copying values from one array to another (which doesn’t allocate at all)
(Also this doesn’t answer the question to interpret a byte array to int, not the other way around)
If you put it in a function I think it does not allocate anymore as it gets optimized.
julia> function test!(a,b)
a_uint8 = reinterpret(UInt8, a)
b_uint8 = reinterpret(UInt8, b)
unsafe_copyto!(pointer(b_uint8), pointer(a_uint8), min(length(b_uint8), length(a_uint8)))
nothing
end
julia> a = rand(UInt8,4)
4-element Vector{UInt8}:
0x6f
0x9b
0x5b
0x64
julia> b = zeros(UInt16,2)
2-element Vector{UInt16}:
0x0000
0x0000
julia> @timev test!(a,b)
0.000001 seconds
elapsed time (ns): 1380
gc time (ns): 0
bytes allocated: 0
pool allocs: 0
non-pool GC allocs: 0
minor collections: 0
full collections: 0
julia> b
2-element Vector{UInt16}:
0x9b6f
0x645b
4 Likes
You are right, as memcpy
also doesn’t allocate that way
using BenchmarkTools
function test()
source = reinterpret(UInt8, Int64[10,11])
target = zeros(Int64, 2)
@btime ccall(:memcpy, Ptr{Cvoid}, (Ptr{Cvoid}, Ptr{Cvoid}, Csize_t), $target, $source, length($source))
end
test()
Or
@ccall memcpy(pointer(target)::Ptr{Cvoid}, pointer(source)::Ptr{Cvoid}, length(source)::Cssize_t)::Cvoid
A bit neater
DNF
January 28, 2023, 8:27pm
12
This is just an artefact of your benchmarking code. It does not allocate. Benchmark like this
@btime b = reinterpret(UInt8, $target)
2 Likes
Yeah you are right, realized that too , the answer then is actually simply:
b = reinterpret(Int64, source)
where I don’t have to fill target
at all
Would such a thing be acceptable?
target .= source.parent
or
target = source.parent
1 Like
Looking at this again, this only works after the reinterpret?
Yes. Is not that your case?
I just used the reinterpret to create the example vector, the actual source
array are just bytes from a file