# Assignment of a \`missing\` value fails in DataFrames 0.11.1

**URL:** <https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379>\
**Category:** Data\
**Created:** [November 29, 2017, 4:42am UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379 "2017-11-29T04:42:09Z")\
**Posts on this page:** 9\
**Page:** 2

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [November 29, 2017, 8:50pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/21 "2017-11-29T20:50:47Z")

</div>

> [@pdeffebach](#):
>
> Could you clarify what you mean by surprising?
> 
> From the user’s perspective, as long as missing values are propagated in a reasonable way, this wouldn’t be unexpected.

I just mean that AFAIK there’s no situation in Julia where an assignment can change the type of the container. When you assign a `String` to a `Vector{Int}`, it doesn’t turn into a `Vector{Any}`.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 30, 2017, 2:09am UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/22 "2017-11-30T02:09:03Z")

</div>

I think that at the very least this thread shows that there is a contingent that would appreciate automatic promotion.

> [@nalimilan](#):
>
> I just mean that AFAIK there’s no situation in Julia where an assignment can change the type of the container. When you assign a String to a Vector{Int}, it doesn’t turn into a Vector{Any}.

The purpose of a library like DataFrames is to abstract from these behaviors, and I think the user would not be concerned by this behavior of columns.

Thanks for all the hard work on DataFrames! I’m excited to start exploring it’s new features.

---

<div class="post-metadata">

**Author:** ![nickeubank](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nickeubank/32/3878_2.png) [@nickeubank](https://discourse.julialang.org/u/nickeubank)\
**Post date:** [November 30, 2017, 3:41pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/23 "2017-11-30T15:41:58Z")

</div>

> [@nalimilan](#):
>
> I just mean that AFAIK there’s no situation in Julia where an assignment can change the type of the container. When you assign a String to a Vector{Int}, it doesn’t turn into a Vector{Any}.

Right, right – that’d probably wreak havoc on type stability.

Well, I’m still in favor of defaulting to `Union{Missings, OTHER_TYPE}` for DataFrames (especially if @StefanKarpinski’s suggestion of using same memory is workable), but I will also admit learning of the `allowmissing!(df)` function does reduce my sense of its importance.

(Side note: as @pdeffebach pointed out: you DataFrame developers are doing amazing work – thanks so much for everything you’re doing!)

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [November 30, 2017, 4:00pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/24 "2017-11-30T16:00:18Z")

</div>

> [@pdeffebach](#):
>
> The purpose of a library like DataFrames is to abstract from these behaviors, and I think the user would not be concerned by this behavior of columns.

I for one am _absolutely_ concerned with the behavior of columns. Any time you want to plug the column into a function or do anything at all with it, you potentially have to worry about this.

Now if not performing the promotion interfered with the “normal” operation of the dataframe, such as groupby’s, join’s, stacking, I’d agree there is something wrong; but one of the greatest features of Julia dataframes is that you _don’t need specialized data structures_ to do this. Nothing is stopping you from putting any type of vector you want into the dataframe, but the default behavior should be to do nothing.

You may find that after spending more time using DataFrames you will come around to this way of thinking. When I first started using it, I’m not sure I had an opinion on this, but after having survived pandas, DataArrays, NullableArrays, I am completely convinced that DataFrames ability to use any `AbstractVector` object and to use simple `Vector`s by default is one of its “killer” features.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 30, 2017, 10:47pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/25 "2017-11-30T22:47:25Z")

</div>

I would also like to add that the addition of `allowmissing!(df)` is great and solves all of these problems. With piping it is super easy to add that at the top of your code.

Once keyword arguments get finalized, maybe adding an option `AllowMissings` in the `DataFrame` constructor would be great, although this would probably involve more importing functions as well.

Regardless, thanks for the new function. This makes everything much easier.

---

<div class="post-metadata">

**Author:** ![Nosferican](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nosferican/32/9275_2.png) [@Nosferican](https://discourse.julialang.org/u/Nosferican)\
**Post date:** [November 30, 2017, 11:44pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/26 "2017-11-30T23:44:36Z")

</div>

Is there `notallowmissing!(df; cols)` or such in the works? I had coded an `allowmissing!` which I am happy to change for calling the DataFrames implementation. However, I would like have the opposite as well… especially helpful after calling `dropmissing!`.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 1, 2017, 4:28pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/27 "2017-12-01T16:28:30Z")

</div>

The name should probably be `disallowmissing!` for such a function.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [December 1, 2017, 4:35pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/28 "2017-12-01T16:35:54Z")

</div>

The name is getting kind of long, but if somebody wants to make a PR to add `disallowmissing!`, why not (should be quite easy). Also if somebody has an idea for a shorter/better replacement of the term “nullable”… 🙂

---

<div class="post-metadata">

**Author:** ![d3vil0p3r](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/d3vil0p3r/32/4379_2.png) [@d3vil0p3r](https://discourse.julialang.org/u/d3vil0p3r)\
**Post date:** [June 28, 2018, 7:15pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379/29 "2018-06-28T19:15:39Z")

</div>

This is an absolute lifesaver. Thanks a lot @nalimilan for this fix 👍

[Previous page](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379.md?page=1)