# Nullables - why? and how?

**URL:** https://discourse.julialang.org/t/nullables-why-and-how/7852
**Category:** New to Julia
**Created:** [December 19, 2017, 4:26pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852 "2017-12-19T16:26:07Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [December 19, 2017, 4:26pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/1 "2017-12-19T16:26:07Z")

</div>

Apologies for what might be a daft question, but I’ve lost touch with the developments in the Julia data wrangling ecosystem over the last year or so and found myself a little puzzled when trying to use DataFrames today.

Going though some online resources it seemed to me that CSV.jl is the preferred I/O method for DataFrames these days so I went ahead and did:

```
using CSV, DataFrames

df = CSV.read("mydata.csv")

```

After which I found myself with a DataFrame populated by objects of varying interesting types, including `Nullable{WeakRefString{UInt8}}`, `Nullable{Float64}`, and others.

I since read up on Nullable.jl and WeakRefString.jl and have some understanding of the motivation behind these types, but I’m still asking myself why this is the default behaviour and how to actually work with this?

Reading through the latest [DataFrames.jl docs](http://juliadata.github.io/DataFrames.jl/latest/index.html) and the [DataFrames section of the Introducing Julia wikibook](https://en.wikibooks.org/wiki/Introducing_Julia/DataFrames), I can’t find the `Nullable` (or `WeakRefString`) type discussed, and hence struggle to understand what the intended workflow is given that a lot of operations don’t seem to be defined on these types.

---

<div class="post-metadata">

### Author: ![amellnik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amellnik/32/137_2.png) [@amellnik](https://discourse.julialang.org/u/amellnik)
#### Post date: [December 19, 2017, 4:29pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/2 "2017-12-19T16:29:49Z")

</div>

What version of DataFrames are you using? You may need to update it. Prior to v0.11 you can use `readtable` to get an older-style `DataFrame` that uses `DataArrays` as columns.

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [December 19, 2017, 4:34pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/3 "2017-12-19T16:34:54Z")

</div>

You may find this useful:

> [@DataFrames 0.11 released](https://discourse.julialang.org/t/dataframes-0-11-released/7296):
>
> After a long and complex development period, we are glad to announce that version 0.11.0 of DataFrames has been released. Among other features listed in the [release notes](https://github.com/JuliaData/DataFrames.jl/releases/tag/v0.11.0), the major change introduced by this version is the move from the NA value (from the [DataArrays](https://github.com/JuliaStats/DataArrays.jl) package) to the new missing value (from the [Missings](https://github.com/JuliaData/Missings.jl) package, and [soon in Base](https://github.com/JuliaLang/julia/pull/24653)). DataFrames have been completely decoupled from DataArrays: the DataFrame constructor will no longer convert columns to DataArrays, but will keep them a…

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [December 19, 2017, 4:49pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/4 "2017-12-19T16:49:23Z")

</div>

I’m on v0.10.1 (I did Pkg.update(), but it seems that some other packages I have installed require 0.10.1 as per the release announcement linked by Tamas?).

I’ve worked out with the help of other forum posts that I can do `weakrefstrings=false` and `nullable=false` to get “normal” data in my DataFrame after reading from csv, but that doesn’t really answer my question. Presumably there’s a reason for why these options are the default behaviour, I’m just struggling to understand the benefits at this point.

Tamas, thanks for linking the release announcement which explains a bit of the background. It does however not mention `Nullable` (unless there’s a connection between `Nullable` and `NA` / `missing` which I’m missing!) - is there a reason for why this isn’t discussed at all in the docs?

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [December 19, 2017, 4:54pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/5 "2017-12-19T16:54:10Z")

</div>

`Nullable` is the earlier attempt that has been phased out. Search the forums for history.

I would suggest that you start using `v0.11.2` of `DataFrames`, which is much nicer. See the topic I linked for removing packages that hold it at `v0.10.1`.

---

<div class="post-metadata">

### Author: ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)
#### Post date: [December 19, 2017, 6:05pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/6 "2017-12-19T18:05:41Z")

</div>

…or use `readtable` until you’re able to move to DataFrames 0.11 (but be careful about reading the docs about the version you are using).

---

<div class="post-metadata">

### Author: ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [December 19, 2017, 7:24pm UTC](https://discourse.julialang.org/t/nullables-why-and-how/7852/7 "2017-12-19T19:24:45Z")

</div>

You can also use [https://github.com/davidanthoff/CSVFiles.jl](https://github.com/davidanthoff/CSVFiles.jl). It will return the “right” missing values in a `DataFrame` no matter on which version of DataFrames.jl you are. So if you are on DataFrames v0.10 it will use the old style `DataArray` for missing values, on DataFrames v0.11 it will create a `DataFrame` that uses the new `Missings` story. If you load data into some other structure it will also give you the right missing values story for that structure (for example `DataValue` in an `IndexedTable` etc.)
