# Adding two Array{Union{Missing,Float64}} returns Array{Float64}, is this by design?

**URL:** <https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472>\
**Category:** Internals & Design\
**Tags:** arrays, missing-values\
**Created:** [October 9, 2021, 11:20am UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472 "2021-10-09T11:20:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 11:20am UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/1 "2021-10-09T11:20:05Z")

</div>

```julia
y = Array{Union{Missing,Float64}}(undef,100);
y[:] = rand(100)
ϵ = Array{Union{Missing,Float64}}(undef,100);
ϵ[:] = rand(100)

julia> eltype(y.+ϵ)
Float64

```

I am not sure but I imagine the idea behind doing this is returning the type that simplifies the array and will perform best after an operation.

Whatever the reason, I wonder what would be the best way to keep the type `Union{Missing,T}` when operating with Arrays regardless whether the Array contains missing values or not.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 9, 2021, 11:46am UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/2 "2021-10-09T11:46:52Z")

</div>

Isn’t that what happens if the assignments are made using dots?

```julia
y .= rand(100)

```

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 11:59am UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/3 "2021-10-09T11:59:55Z")

</div>

> [@rafael.guerra](#):
>
> `y .= rand(100)`

Oh! that’s nice!

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 9, 2021, 12:00pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/4 "2021-10-09T12:00:47Z")

</div>

The types do not seem to be preserved after the sum (Julia 1.6), though.

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:01pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/5 "2021-10-09T12:01:42Z")

</div>

> [@rafael.guerra](#):
>
> The types do not seem to be preserved after the sum though.

Yeah, that’s a bit of a problem, I was wondering what was the Julia way to preserve the type with missing values.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 9, 2021, 12:04pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/6 "2021-10-09T12:04:43Z")

</div>

Not sure if this is recommended:  
`typeof(y)(y+ϵ)`  
or:  
`Union{typeof(y),typeof(ϵ)}(y+ϵ)`

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:05pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/7 "2021-10-09T12:05:48Z")

</div>

> [@rafael.guerra](#):
>
> Not sure if this is recommended:  
> `typeof(y)(y+ϵ)`

In the package `Missing.jl` they use `convert` but I was wondering if there was a better way

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [October 9, 2021, 12:07pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/8 "2021-10-09T12:07:07Z")

</div>

It’s because your arrays don’t actually contain any `missing` values. If you do `y[end] = missing`, the return type is `Vector{Union{Missing, Float64}}`:

```julia
julia> y[end] = missing
missing

julia> typeof(y .+ ϵ)
Vector{Union{Missing, Float64}}

```

I haven’t checked what happens in a function, but I suspect this is a global-scope only thing.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [October 9, 2021, 12:15pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/9 "2021-10-09T12:15:52Z")

</div>

`allowmissing` in Missings.jl would be the best way.

> [@Sukera](#):
>
> I haven’t checked what happens in a function, but I suspect this is a global-scope only thing.

Seems to exist in functions too.

I think this makes sense from the design of broadcasting. But it is annoying.

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:23pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/10 "2021-10-09T12:23:07Z")

</div>

> [@Sukera](#):
>
> `julia> y[end] = missing`

Yeah, I know, but problem is that if I do `z = y .+ ϵ` then `z[end] = missing` throws an error, which is what I want to avoid.

> [@pdeffebach](#):
>
> `allowmissing` in Missings.jl would be the best way.

This is what `allowmissing` does in `Missing.jl`:

`allowmissing(x::AbstractArray{T}) where {T} = convert(AbstractArray{Union{T, Missing}}, x)`

I could convert directly instead loading a package but I was wondering if this is supposed to be the default behavior, I am asking because for certain algorithms I will have to constantly convert for every single operation I do with Arrays capable of containing missing values.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 9, 2021, 12:42pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/11 "2021-10-09T12:42:29Z")

</div>

> [@viraltux](#):
>
> `z = y .+ ϵ`

The right-hand side allocates a new vector, as such, its type may need shrinking or widening. In this case, the algorithm decided shrinking is useful (makes sense, because you didn’t have any missing to begin with)

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:47pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/12 "2021-10-09T12:47:36Z")

</div>

> [@jling](#):
>
> makes sense, because you didn’t have any missing to begin with

I would say in analytics that’s debatable; precisely the whole point of defining Array{Union{Missing,T}} is because I am planing to insert missing values but I dot not want any type conversion every time my Arrays happen not to have any in an operation.

The workaround I am considering is using `NaN` isntead `missing` since `NaN isa Number` is `true` and I would not run into this kind of problems.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 9, 2021, 12:50pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/13 "2021-10-09T12:50:17Z")

</div>

> [@viraltux](#):
>
> to insert missing values

then you should be doing

```julia
y .+= ϵ

```

instead? and keep using `y` since that’s your “pre-allocated vector”

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:53pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/14 "2021-10-09T12:53:55Z")

</div>

> [@jling](#):
>
> then you should be doing `y .+= ϵ` instead?

That’s nice, as long as I don’t need `y` for anything else…

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 9, 2021, 12:54pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/15 "2021-10-09T12:54:51Z")

</div>

z = copy(y)  
z .= y + ϵ

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 9, 2021, 12:55pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/16 "2021-10-09T12:55:06Z")

</div>

well, you can also `z = (copy(y) .+= ϵ)`

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:56pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/17 "2021-10-09T12:56:11Z")

</div>

> [@rafael.guerra](#):
>
> z = copy(y)  
> z .= y .+ ϵ

This one perhaps could be faster with

```julia
z = similar(y)
z .= y .+ ϵ

```

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 12:57pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/18 "2021-10-09T12:57:11Z")

</div>

Missing values are so important that I wonder if we could ask for them to be a subtype of Number.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [October 9, 2021, 1:02pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/19 "2021-10-09T13:02:05Z")

</div>

definitely not

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [October 9, 2021, 1:43pm UTC](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472/20 "2021-10-09T13:43:09Z")

</div>

> [@jling](#):
>
> definitely not

Very useful explanation

[Next page](https://discourse.julialang.org/t/adding-two-array-union-missing-float64-returns-array-float64-is-this-by-design/69472.md?page=2)