# Should Generators finally be given eltype

**URL:** <https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507>\
**Category:** Internals & Design\
**Created:** [February 6, 2019, 2:44pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507 "2019-02-06T14:44:43Z")\
**Posts on this page:** 6\
**Page:** 2

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [February 7, 2019, 4:49pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/21 "2019-02-07T16:49:13Z")

</div>

> [@yuyichao](#):
>
> AFAICT most of the code in there is actually dealing with iterator, i.e. the customized and non-reusable part, rather than dealing with widening.

The interface I am thinking about is for _collections_, ie

```julia
container2 = store!_or_widen(container, elt)

```

defined for various types of `container`, complete with a `empty_container(::Type{T})` and a `finalize_container(container)`. When `container2 !=== container`, branch to recursion.

Eg the obvious one is `Vector{T}`, created by `Base.Bottom[]` (which I think is what should be returned for empty collections into vectors), and finalized as itself.

You are right that the _application_ of this interface is fairly trivial, I was thinking of standardizing the container part. I am experimenting with this in [FunctionalTables.jl](https://github.com/tpapp/FunctionalTables.jl), but it is heavily WIP.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [February 7, 2019, 5:02pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/22 "2019-02-07T17:02:16Z")

</div>

> [@cstjean](#):
>
> Maybe I just need to assume that I can’t rely on the type of `f.(v)` at all, and code accordingly.

Yes. Without any context or without a well defined inference rule, we just can’t give any guarantee on the return type of `f.(v)` other than the current behavior. I’ll even say that **in some sense** a well understood dependency on the input is better to account for or debug than something that only happens when the inference decide to give you a bad day…

In this case, the right way to give it the context is to manually give it the type since you know what you are expecting (you have code that want to throw in a `NaN` of type `Float64` after all).

> [@Tamas\_Papp](#):
>
> You are right that the _application_ of this interface is fairly trivial, I was thinking of standardizing the container part. I am experimenting with this in [FunctionalTables.jl](https://github.com/tpapp/FunctionalTables.jl), but it is heavily WIP.

Right. Having some wrappers around the basic building block is certainly fine, especially in a package. (I wouldn’t personally call those building block any more since they’ll make more assumption than the “building blocks” in Base). My warning would be that such an interface might encourage people to match that interface when they shouldn’t, e.g. for different array types or even container type, like `Dict`/`Set`. This warning would not apply as much anymore once you’ve got a more complete collection of utilities written for different situations/container types and such a collection is certainly useful (at that point I’ll almost certainly not call it building blocks anymore but that’s not important…)

---

<div class="post-metadata">

**Author:** ![ndinsmore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ndinsmore/32/7433_2.png) [@ndinsmore](https://discourse.julialang.org/u/ndinsmore)\
**Post date:** [February 7, 2019, 5:31pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/23 "2019-02-07T17:31:11Z")

</div>

> [@cstjean](#):
>
> I’m saying that the fewer branches to test, the better. `f.(::Vector{Union{Missing, Float64}})` gives me four possible outcomes to worry about.

I can only figure out what three of the four branches you are talking about are.

But for what it is worth I use the signature `AbstractArray{Union{Float64,T}} where T<:Missing` to match two cases but not the third:

```julia
Vector{Float64} <: AbstractArray{Union{Float64,T}} where T<:Missing == true 
Vector{Union{Float64,Missing}} <: AbstractArray{Union{Float64,T}} where T<:Missing == true
Vector{Missing} <: AbstractArray{Union{Float64,T}} where T<:Missing == false

```

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [February 7, 2019, 5:31pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/24 "2019-02-07T17:31:57Z")

</div>

I’ve been wondering how hard it would be to implement 0.4-style comprehensions and broadcasting, as a package-provided macro. Something like `@stable f.(x)`, and `@stable (f(x) for x in v)`, that returns a `StableGenerator{Base.return_type(...)}`, so it has an `eltype`

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 7, 2019, 8:06pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/25 "2019-02-07T20:06:29Z")

</div>

`BroadcastArray` does that [https://github.com/JuliaArrays/LazyArrays.jl](https://github.com/JuliaArrays/LazyArrays.jl)

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 7, 2019, 8:22pm UTC](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507/26 "2019-02-07T20:22:08Z")

</div>

> [@cstjean](#):
>
> ```julia
> function kalman_filter(initial_state, observations, theta, N)
> filtered = []
> likelihoods = []
> state = initial_state
> for i in 1:N
> state, ll = kernel(state, observations[i], theta)
> push!(filtered, state)
> push!(likelihoods, ll)
> end
> return filtered, likelihoods
> end
> 
> ```
> 
> How should I pick the eltype of `filtered` ?

It would be nice if we can just write

```julia
function kalman_filter(initial_state, observations, theta, N)
   filtered = Union{}[]
   likelihoods = Union{}[]
   state = initial_state
   for i in 1:N
       state, ll = kernel(state, observations[i], theta)
       filtered = push!!(filtered, state)
       likelihoods = push!!(likelihoods, ll)
   end
   return filtered, likelihoods
end

```

where `push!!` is a hypothetical `push!`-or-widen API. Is it crazy to imagine that compiler can do the “union-splitting” the for loop and lower it to a type stable code (when `kernel` is well behaved) in near future?

[Previous page](https://discourse.julialang.org/t/should-generators-finally-be-given-eltype/20507.md?page=1)