Difference between `Union{Missing, Float64}` and `Float64?`

New to Julia
1

I have a DataFrame that was created by importing a CSV. Then I applied several transformations. Now when I run describe on it I get some columns that report Union{Missing, Float64} as type and others report Float64?.

What is the difference? Both columns have Float64 type and contain missing values.

2

you can’t possibly have missing values in the Float64 column. But you certainly can have columns of Union{Missing, Float64} but really no longer have any missings, in which case you can use:
disallowmissing!

1 Like
3

The column type is Float64? with the question mark at the end.

4

if you see a T? with a question mark that means Union{Missing, T}. As described in the Getting Started section of docs: Getting Started · DataFrames.jl

1 Like
5

Ok, so they are the same.
It’s confusing that two number columns that contain missing values are marked one as Union{Missing, Float64} and the other column as Float64?

6

That does sound strange, where is this happening? In the REPL I see:

julia> df = DataFrame(rand([missing; 1.0:10.0], 10, 10), :auto)
10×10 DataFrame
 Row │ x1         x2        x3        x4        x5        x6         x7         x8         x9         x10      
     │ Float64?   Float64?  Float64?  Float64?  Float64?  Float64?   Float64?   Float64?   Float64?   Float64? 
─────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │       8.0       7.0       1.0       8.0       3.0        3.0        1.0  missing          3.0       9.0
7

Good question.
I had some time today to investigate and came up with the following conclusion: it’s just a matter of display real estate.
If the column name is short, then there would not be enough space to fit the Union{...} header

julia> DataFrame(a = [1,2,3,missing])
4×1 DataFrame
│ Row │ a       │
│     │ Int64?  │
├─────┼─────────┤
│ 1   │ 1       │
│ 2   │ 2       │
│ 3   │ 3       │
│ 4   │ missing │

However, if the column name is long enough, then the Union{...} type name representation would fit and so would be used:

julia> DataFrame(a12345678912345678901 = [1,2,3,missing])
4×1 DataFrame
│ Row │ a12345678912345678901 │
│     │ Union{Missing, Int64} │
├─────┼───────────────────────┤
│ 1   │ 1                     │
│ 2   │ 2                     │
│ 3   │ 3                     │
│ 4   │ missing               │
5 Likes