Should Generators finally be given eltype

The interface I am thinking about is for collections, ie

container2 = store!_or_widen(container, elt)

defined for various types of container, complete with a empty_container(::Type{T}) and a finalize_container(container). When container2 !=== container, branch to recursion.

Eg the obvious one is Vector{T}, created by Base.Bottom[] (which I think is what should be returned for empty collections into vectors), and finalized as itself.

You are right that the application of this interface is fairly trivial, I was thinking of standardizing the container part. I am experimenting with this in FunctionalTables.jl, but it is heavily WIP.

1 Like

Yes. Without any context or without a well defined inference rule, we just canā€™t give any guarantee on the return type of f.(v) other than the current behavior. Iā€™ll even say that in some sense a well understood dependency on the input is better to account for or debug than something that only happens when the inference decide to give you a bad dayā€¦

In this case, the right way to give it the context is to manually give it the type since you know what you are expecting (you have code that want to throw in a NaN of type Float64 after all).

Right. Having some wrappers around the basic building block is certainly fine, especially in a package. (I wouldnā€™t personally call those building block any more since theyā€™ll make more assumption than the ā€œbuilding blocksā€ in Base). My warning would be that such an interface might encourage people to match that interface when they shouldnā€™t, e.g. for different array types or even container type, like Dict/Set. This warning would not apply as much anymore once youā€™ve got a more complete collection of utilities written for different situations/container types and such a collection is certainly useful (at that point Iā€™ll almost certainly not call it building blocks anymore but thatā€™s not importantā€¦)

I can only figure out what three of the four branches you are talking about are.

But for what it is worth I use the signature AbstractArray{Union{Float64,T}} where T<:Missing to match two cases but not the third:

Vector{Float64}                   <: AbstractArray{Union{Float64,T}} where T<:Missing   == true 
Vector{Union{Float64,Missing}}    <: AbstractArray{Union{Float64,T}} where T<:Missing   == true
Vector{Missing}                   <: AbstractArray{Union{Float64,T}} where T<:Missing   == false

Iā€™ve been wondering how hard it would be to implement 0.4-style comprehensions and broadcasting, as a package-provided macro. Something like @stable f.(x), and @stable (f(x) for x in v), that returns a StableGenerator{Base.return_type(...)}, so it has an eltype

BroadcastArray does that https://github.com/JuliaArrays/LazyArrays.jl

2 Likes

It would be nice if we can just write

function kalman_filter(initial_state, observations, theta, N)
   filtered = Union{}[]
   likelihoods = Union{}[]
   state = initial_state
   for i in 1:N
       state, ll = kernel(state, observations[i], theta)
       filtered = push!!(filtered,  state)
       likelihoods = push!!(likelihoods, ll)
   end
   return filtered, likelihoods
end

where push!! is a hypothetical push!-or-widen API. Is it crazy to imagine that compiler can do the ā€œunion-splittingā€ the for loop and lower it to a type stable code (when kernel is well behaved) in near future?

3 Likes