Reinterpet fails on empty array

Reinterpreting an empty array fails.

I don’t know why A fails, while it is clear that B will succeed:

julia> struct A end

julia> reinterpret(A, Int8[])
ERROR: DivideError: integer division error
Stacktrace:
 [1] rem
   @ ./int.jl:285 [inlined]
 [2] reinterpret(#unused#::Type{A}, a::Vector{Int8})
   @ Base ./reinterpretarray.jl:42
 [3] top-level scope
   @ REPL[3]:1

julia> struct B
       a::UInt8
       b::Int8
       end

julia> reinterpret(B, UInt8[1,1])
1-element reinterpret(B, ::Vector{UInt8}):
 B(0x01, 1)

julia> reinterpret(B, UInt8[1,255])
1-element reinterpret(B, ::Vector{UInt8}):
 B(0x01, -1)

How would you reinterpret something to nothing? The array holds Int8, which are one byte large. A is an empty singleton struct - its size is zero:

julia> struct A end

julia> sizeof(A)
0

The length of UInt8[] is also 0, so there is only one way to reinterpret it to nothing: doing nothing.

I really don’t want to sound obtuse, it’s just that I don’t see the problem here.

julia> length(UInt8[])
0

Maybe this helps:

julia> struct A end

julia> as = [ A() for _ in 1:10 ]
10-element Vector{A}:
 A()
 A()
 A()
 A()
 A()
 A()
 A()
 A()
 A()
 A()

julia> sizeof(as)
0 # in bytes

julia> sizeof([ Int8(0) for _ in 1:10 ]) 
10 # in bytes

How would you reinterpret the 10 bytes taken up by those Int8 to the 0 bytes taken up by the As? It’s not per-se about the length of each array, but rather what has to happen to the underlying memory, as that’s what reinterpret is doing - reinterpreting how you interpret a piece of memory.

I’m deliberately showing arrays with elements in them here, as that’s something that has to be preserved when reinterpreting. The result of reinterpret is linked to the original array after all.

I believe the confusion here might comes from the fact that reinterpret works on the element sense:

julia> reinterpret(B, 1:10) # 1:10 is a lazy iterator
40-element reinterpret(B, ::UnitRange{Int64}):
...

For reinterpret(A, UInt8[]), since there’s no element in the array block, it’s understandable that it fails. --we might need some extra check to throw meaningful error messages, though. Making it a zero-sized ReinterpretedArray is also a valid result to me.

Yep, but in your example you do actually have numbers in your Int8[] arrays. Whereas, in my original example, the array itself is empty, so it can be readily reinterpreted, can’t it?

No, because the original array may still be modified:

julia> arr = UInt16[]
UInt16[]

julia> re_arr = reinterpret(UInt8, arr)
0-element reinterpret(UInt8, ::Vector{UInt16})

julia> push!(arr, 1)
1-element Vector{UInt16}:
 0x0001

julia> re_arr
2-element reinterpret(UInt8, ::Vector{UInt16}):
 0x01
 0x00

Which breaks down when the thing you reinterpreted to has sizeof(A) == 0.

Consider also the docstring:

help?> reinterpret
search: reinterpret

  reinterpret(type, A)

  Change the type-interpretation of a block of memory. For arrays,
  this constructs a view of the array with the same binary data as
  the given array, but with the specified element type.

Would you take a block of memory with length 0 as a block of memory (specifically in the context of reinterpret)?

1 Like

I think I would, since the object I am trying to “view” also has size 0, I think.

Anyway, if everyone agrees this is how it is supposed to behave, which is understandable, maybe some checks to throw a more meaningful error as @JohnnyChen94 suggested might be of use to people who encounters this. Should I raise an issue in GitHub?

The object itself at time of reinterpreting does, yes, but that’s not the same as keeping that size over the lifetime of the object. reinterpreting empty arrays around seems rather odd to me, since you won’t be able to use them properly with elements inside of them. It’s similar to how padding has to agree for different structs to be allowed to be reinterpreted.

May I ask, how did you come across this?

Yes, that’s a good idea. Probably a check that the type to be reinterpreted to has sizeof != 0 (or only allow that if both source & target type have sizeof == 0)?

2 Likes

I am writing a Julia implementation of a protocol. Some of the messages of the protocol have size 0, since they have no contents further than a header (which is common to other types). However, it is still useful to know what type of message it was, since it can be used for dispatching.

Sounds like you want an Enum, but for more details I’d have to know more about the protocol & your implementation in question.

As it turns out, that already works:

julia> struct A end

julia> struct B end

julia> reinterpret(B, A[])
0-element reinterpret(B, ::Vector{A})

I tried with a Tuple, but reinterpret works different for it.

Since I am reading from a stream, literally I am reading 0 bytes, so I had hoped that I could reinterpret it as I do for the other types (which all have a layout that is compatible with reinterpretation).

How would an Enum solve my problem here? By the way, I have literally 100 types of different messages

Ah, I see! Yes, an enum won’t help then. I don’t know what you’re doing with the read & reinterpreted block of memory, but I think reinterpret to empty singleton types is dangerous - you don’t have a link between the raw array and the resulting reinterpret anymore, as there’s nothing prevent the underlying array from changing. There’s also nothing to define how long the reinterpreted array should be, as that’s always tied to the size of the parent array.

I did find this though:

julia> struct A end

julia> arr = Int8[01,2,3]
3-element Vector{Int8}:
 1
 2
 3

julia> arrv = @view arr[1:0]
0-element view(::Vector{Int8}, 1:0) with eltype Int8

julia> reinterpret(A, arrv)
0-element reinterpret(A, view(::Vector{Int8}, 1:0))

which may work for you? The view is immutable, so won’t suddenly change where it looks or how long it is, so it should be safe. Is probably also safe for your other types, as long as the underlying data/array doesn’t suddenly vanish.

I am sorry, but this errors me on an empty REPL…

Which version are you on? Also, what’s the error? This should then work in the upcoming 1.8:

julia> versioninfo()
Julia Version 1.8.0-beta3
Commit 3e092a2521* (2022-03-29 15:42 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 4 on 4 virtual cores
Environment:
  JULIA_NUM_THREADS = 4
julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635 (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

Hm, from the source of reinterpret I can’t tell what the error would be :thinking: What error do you get?

The same as before,

julia> reinterpret(A, arrv)
ERROR: DivideError: integer division error
Stacktrace:
 [1] rem
   @ ./int.jl:285 [inlined]
 [2] reinterpret(#unused#::Type{A}, a::SubArray{Int8, 1, Vector{Int8}, Tuple{UnitRange{Int64}}, true})
   @ Base ./reinterpretarray.jl:42
 [3] top-level scope
   @ REPL[6]:1

I installed the beta version now and it does work, thanks. Maybe it’s something about view that has changed for 1.8?

Still, finding a solution for Julia < 1.8 would be good, if possible.

Aren’t you basically arguing that 0/0 should be 0? I’m not conformable with that.

Also, this seems like something that should fail on the type level, and not work as a special case for some values.