How to change the type of a column of a DataFrame

General Usage
1

I have a DataFrame that has a few columns of the types “Union{Missing, String}” and “Union{Missing, Float64}”. But there are no missing values in the data.

I would like to change the type of these columns to either “String” or “Float64”. How can I achieve that?

1 Like
2

disallowmissing e.g.

using Missings

a = DataFrame(a = Union{Missing, String}["a", "b"])
a[!, :a] = disallowmissing(a[!, :a])

The reverse of disallowmissing is (you guessed it!) allowmissing

2 Likes
3

Thank you for your explanation!

I couldn’t find this info in the manual: Missing Values · The Julia Language

4

There was a long story. Anyway, these fucntions are in https://github.com/JuliaData/Missings.jl

Perhaps they should be part of base, but it’s in an external package

1 Like
5

In this same dataframe that allows missing
a = DataFrame(a = Union{Missing, String}["a", "b"])
if I try push!(a,[nothing])
I get an error: Cannot convert an object of type Nothing to an object of type String
It is a missing data, should missing allow nothing as well?

6

Typically nothing should not be used to represent missing data in the statistical sense.

7

Thanks @Tamas_Papp
I’m trying to push data from a JSON I got through a request which consist of nothing, I have to convert nothing in the JSON to missing before I can push it in the dataframe.
I converted the dataframe to accept any and push it and then replaced nothing with missing.
I was wondering if there was a better way.

8

You can also do:

julia> a = DataFrame(a = Union{Missing, String}["a", "b"])
2×1 DataFrame
 Row │ a       
     │ String? 
─────┼─────────
   1 │ a
   2 │ b

julia> disallowmissing!(a, :a)
2×1 DataFrame
 Row │ a      
     │ String 
─────┼────────
   1 │ a
   2 │ bhich consist of nothing, I have to convert nothing in the JSON to missing before I can push it in the dataframe.

which is a bit more flexible as it allows you to pass any column selector and has error keyword argument.hich consist of nothing, I have to convert nothing in the JSON to missing before I can push it in the dataframe.

If you are push!-ing data to the data frame use promote=true to perform auto-promotion of column eltypes (although it would be better to do the conversion before pushing as @Tamas_Papp suggested).

1 Like
9

I had the same problem recently. I tried to use transform!, but I couldn’t make it work. I ended up using dropmissing!.

Do you have comments regarding using these two other functions for this task?

10

transform! would do it, e.g.:

transform!(df, :a => disallowmissing => :a)

but dropmissing! has error kwarg and it was introduced in DataFrames.jl before transform! was implemented.

2 Likes