Eltypes of `view`s

General Usage Performance
1

Let’s say I have a vector t = [1, 2, missing]. And I take sub_t = view(t, 1:2). I know that no elements sub_t contain missing values.

Is there any way to make the eltype of sub_t be Int instead of Union{Int, Missing}?

2

since compiler can’t know, maybe you are looking for a way to remove Missing from the Union type returned by eltype?

3

Yeah. It’s definitely lying about the internal memory of the object, since the underlying vector still has room for the Missing mask.

I’m thinking of scenarios where functions have overly-restrictive type parameters

foo(x::AbstractVector{Int})
t = [1, 2, missing]
sub_t = view(t, 1:2)
foo(sub_t)

I’m not sure if just overloading the eltype is enough, right? It has to actually be in the type parameter.

1 Like
4

the thing though is that, as I imagine you know:

t = [1,2,missing]
sub_t = view(t, 1:2)  # does not contain missing
t[1] = missing
sub_t[1] === missing

we had a similar issue with ScientificTypes but for slices (so not views), where if you take t[1:2] it would not be of eltype Int. For this you can apply identity.(v) to create a “tight” copy of it something like

t = [1,2,missing]
w = t[1:2]  # has missing type still
z = (eltype(w) >: Missing && findfirst(ismissing, w) === nothing) ?
       identity.(w) : w

not a view though…

1 Like
5

Since a view is quite literally a window into another objects’ memory and the original object can’t loose the ability to hold mixed types for arbitrary indices (it’s a property of the whole object after all), you won’t get rid of this without copying. Setting an index of a view means setting another index in the original object - the view is only there as a thin passthrough and index calculation.

If it were a property of each position however, it would be possible.

1 Like
6

Yeah. That’s true.

If I were to implement my own type for this it would maybe disallow inputting missing values.

But it sounds like re-implementing SubArray or something similar would be the way to go for this, which is probably more work than I would want to go for at the moment.

7

You could also check every covered index for missing and only allow construction if truly all elements are non-missing. That would allow type stability, but doesn’t change the fact that the underlying array still has the Union.

1 Like
8

Yeah that’s exactly what I’m doing, so I’m 100% sure there aren’t any missing values in the Array. This is like what skipmissing does here.

IteratorEltype(::Type{SkipMissing{T}}) where {T} = IteratorEltype(T)
eltype(::Type{SkipMissing{T}}) where {T} = nonmissingtype(eltype(T))
10

This works, but the view is lost?

foo(x::AbstractVector{Int}) = dot(x,x)^3   # some example
t = [1, 2, missing]
sub_t = view(t, 1:2)
foo(sub_t)          # Error
foo(Int.(sub_t))    # OK = 125
11

No. That creates a copy. The point of using a view is that I don’t have to make a copy of the data.

1 Like
12

If you really wanted, you could probably get access to the data array via a pointer. Then you’d just take a view of the data array, as you don’t really care what values live in the masked segments. There isn’t a blessed way to do this (https://github.com/JuliaLang/julia/issues/26681).

1 Like