# Issue with DataFrames, operations on DataFrames now return Nullable Arrays?

**URL:** https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946
**Category:** General Usage
**Created:** [July 19, 2017, 8:16am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946 "2017-07-19T08:16:40Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![IljaK91](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iljak91/32/44301_2.png) [@IljaK91](https://discourse.julialang.org/u/IljaK91)
#### Post date: [July 19, 2017, 8:16am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/1 "2017-07-19T08:16:40Z")

</div>

I would like to change entries in a DataFrame. Before, I was using:

`df2[df2[:country] .== "United States", :country] = "USA"`

which basically looks for entries equal to “United States” in column country and fills those entries if “USA” instead. However, strangely something about the type of `df2[:country] .== "United States"` changed, which is now `NullableArrays.NullableArray{Bool,1}`. I am pretty sure that the last time I used my code it was a DataArray, but I am not 100% sure. Running the line above gives me

```julia
MethodError: no method matching setindex!(::DataFrames.DataFrame, ::String, ::NullableArrays.NullableArray{Bool,1}, ::Symbol)
include_string(::String, ::String) at loading.jl:515
include_string(::String, ::String, ::Int64) at eval.jl:30
include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N} where N) at eval.jl:34
(::Atom.##49#52{String,Int64,String})() at eval.jl:50
withpath(::Atom.##49#52{String,Int64,String}, ::String) at utils.jl:30
withpath(::Function, ::String) at eval.jl:38
macro expansion at eval.jl:49 [inlined]
(::Atom.##48#51{Dict{String,Any}})() at task.jl:80

```

Which I am a bit confused by. My code worked before, df2 is of type DataFrame as it should be, so what changed and how do I need to adapt my code?

---

<div class="post-metadata">

### Author: ![bramtayl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bramtayl/32/3614_2.png) [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [July 19, 2017, 8:21am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/2 "2017-07-19T08:21:16Z")

</div>

How did you read in the data?

---

<div class="post-metadata">

### Author: ![IljaK91](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iljak91/32/44301_2.png) [@IljaK91](https://discourse.julialang.org/u/IljaK91)
#### Post date: [July 19, 2017, 8:23am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/3 "2017-07-19T08:23:04Z")

</div>

Before I used, which throws me now an error connected to PyCall for some reason:

**EDIT: The problem was that for some reason I have to give the full path now, any idea why is that? Just giving the filename worked before. Moreover, the command I used to manipulate the Dataframe works with readxlsheet but not with CSV.read. Is there an explanation for that?**

`df1 = readxlsheet(DataFrame,"JSTdatasetR1.xlsx", "Data")`

Now I use which is also supposed to give me a DataFrame.

`df1 = CSV.read(file; delim=";", types=Dict(21=>Float64))`

Is there an issue with that?

---

<div class="post-metadata">

### Author: ![jonathanBieler](https://avatars.discourse-cdn.com/v4/letter/j/82dd89/32.png) [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [July 19, 2017, 8:36am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/4 "2017-07-19T08:36:50Z")

</div>

If you have no NA’s in your data you can set `nullable=false` in `CSV.read` and it will return plain arrays.

---

<div class="post-metadata">

### Author: ![ValdarT](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/valdart/32/24146_2.png) [@ValdarT](https://discourse.julialang.org/u/ValdarT)
#### Post date: [July 19, 2017, 10:35am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/5 "2017-07-19T10:35:13Z")

</div>

CSV, ODBC and other packages built on top of DataStreams currently use Nullable to handle missing values. This is different from the NA/DataArray approach that DataFrames uses by default. But it is still a DataFrame.  
All this is unfortunate but it is being actively worked on. By Julia 0.7 all the packages should use a new approach that will be easier to work with - see [this announcement](https://discourse.julialang.org/t/announcement-an-update-on-dataframes-future-plans/4646).

In the meantime you can use DataTables that works more naturally with the Nullable data type or convert things manually.

---

<div class="post-metadata">

### Author: ![bramtayl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bramtayl/32/3614_2.png) [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [July 19, 2017, 11:07am UTC](https://discourse.julialang.org/t/issue-with-dataframes-operations-on-dataframes-now-return-nullable-arrays/4946/6 "2017-07-19T11:07:44Z")

</div>

You probably had to include the full path because your working directory was different.
