# Initiate DataFrame variable with missing values and type Union{Missing, Float64}

**URL:** <https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213>\
**Category:** New to Julia\
**Created:** [May 15, 2021, 7:49pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213 "2021-05-15T19:49:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![klwlevy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/klwlevy/32/25272_2.png) [@klwlevy](https://discourse.julialang.org/u/klwlevy)\
**Post date:** [May 15, 2021, 7:49pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/1 "2021-05-15T19:49:49Z")

</div>

How do I best initiate a new variable to an existing DataFrame, if I want the new variable to contain only missing values? Later I want to assign float values to the new variable.

```julia
# Initiate a simple dataframe with one variable (:a) and two rows
df = DataFrame(a=[1, 2])

# Declare a new variable (:weight) and initialize it to missing (Type is Missing, but want Union{Missing, Float64})
df[:, :weight] .= missing

# I can change a single value for variable :a
df[1, :a] = 3

# I cannot change a missing value to a float value, i.e. I cannot change a single value of :weight
#df[1, :weight] = 1.0

# Verbose solution - declare new dataframe with a single column (:weight2) of the correct type
single_col_df = DataFrame(weight2 = Union{Missing, Float64}[missing for i in eachrow(df)])

# Concatenate the two dataframes horizontally
df = hcat(df, single_col_df)

# Now I can change a missing value to a float value
df[1, :weight2] = 1.0

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 15, 2021, 8:44pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/2 "2021-05-15T20:44:44Z")

</div>

There isn’t a way to do this super elegantly. I would do the following

```julia
df[:, :weight] = Union{Missing, Float64}[missing for i in 1:nrow(df)]

```

---

<div class="post-metadata">

**Author:** ![klwlevy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/klwlevy/32/25272_2.png) [@klwlevy](https://discourse.julialang.org/u/klwlevy)\
**Post date:** [May 15, 2021, 9:31pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/3 "2021-05-15T21:31:54Z")

</div>

Thanks a lot for the quick reply.

Yes, it would be nice if one could do it with broadcasting.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 15, 2021, 9:35pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/4 "2021-05-15T21:35:59Z")

</div>

yeah would be cool. Probably wouldn’t be too hard to add (someplace, not in DataFrames)

---

<div class="post-metadata">

**Author:** ![qsong](https://avatars.discourse-cdn.com/v4/letter/q/d07c76/32.png) [@qsong](https://discourse.julialang.org/u/qsong)\
**Post date:** [May 16, 2021, 4:43am UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/5 "2021-05-16T04:43:32Z")

</div>

```julia
df.weight = similar(df.a, Union{Missing, Float64})

```

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [May 16, 2021, 5:53am UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/6 "2021-05-16T05:53:30Z")

</div>

This relies on an implementation detail though, see the note [here](https://docs.julialang.org/en/v1/manual/missing/#Arrays-With-Missing-Values) in the manual. I think the recommended way is:

```julia
df.weight = Vector{Union{Missing,Float64}}(missing, nrow(df))

```

---

<div class="post-metadata">

**Author:** ![qsong](https://avatars.discourse-cdn.com/v4/letter/q/d07c76/32.png) [@qsong](https://discourse.julialang.org/u/qsong)\
**Post date:** [May 16, 2021, 2:53pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/7 "2021-05-16T14:53:26Z")

</div>

Thanks, but it’s a little confusing

> Using `undef` or `similar` … is not the correct way

but it currently give the right result.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [May 16, 2021, 3:18pm UTC](https://discourse.julialang.org/t/initiate-dataframe-variable-with-missing-values-and-type-union-missing-float64/61213/8 "2021-05-16T15:18:22Z")

</div>

Yes it’s a bit confusing… This was discussed in [this PR](https://github.com/JuliaLang/julia/pull/31091) and the follow-up one. Relying on this behavior is a bit like using `reduce` with a non-associative operation, or relying on the order of iteration for `Dict` keys: it might work in one case, but can fail in another:

```julia
julia> module A struct B end end;

julia> similar([1,2,3], Union{Missing, A.B})
3-element Vector{Union{Main.A.B, Missing}}:
 Main.A.B()
 Main.A.B()
 Main.A.B()

```

This reminds me of an anecdote concerning the Go language: the maps (equivalent of Julia Dict) had a non-specified iteration order. But people would sometimes come to rely on it… So they ended up [adding randomization](https://golang.org/doc/go1#iteration) to the iteration 🙂