# Dataframe parses differently if data is passed in columns vs as an array
**URL:** https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130
**Category:** General Usage
**Tags:** dataframes
**Created:** [April 27, 2021, 9:31pm UTC](https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130 "2021-04-27T21:31:23Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@brianguenter](https://discourse.julialang.org/u/brianguenter)
#### Post date: [April 27, 2021, 9:31pm UTC](https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130/1 "2021-04-27T21:31:23Z")
Is it expected behavior that the DataFrames constructor will return different results if the data is passed in by column or as a matrix? In the former case the type of the column is more precisely inferred. In the latter case no type inference appears to happen.
I found an example from 2017 of creating a DataFrame from a matrix. The types were properly inferred in that example. When I execute the example on Julia 1.6.1 the column types are all Any.
Here’s a MWE. In the first case the type of the first column is correctly inferred to be string, the second Float64, and the last Float64?.
In the second case, where the DataFrame is constructed from a matrix, the type of all columns is Any.
```julia
julia> a = [
:Name :Radius :SemiDiameter
"this" 1.0 4.0
"that" 2.0 missing
]
3×3 Matrix{Any}:
:Name :Radius :SemiDiameter
"this" 1.0 4.0
"that" 2.0 missing
julia> df1 = DataFrame(
Name = ["this", "that"],
Radius = [1.0, 2.0],
SemiDiameter = [4.0, missing]
)
2×3 DataFrame
Row │ Name Radius SemiDiameter
│ String Float64 Float64?
─────┼───────────────────────────────
1 │ this 1.0 4.0
2 │ that 2.0 missing
julia> df2 = DataFrame(a[2:end,:],Symbol.(a[1,:]))
2×3 DataFrame
Row │ Name Radius SemiDiameter
│ Any Any Any
─────┼────────────────────────────
1 │ this 1.0 4.0
2 │ that 2.0 missing
julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: AMD EPYC 7702P 64-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, znver2)
Environment:
JULIA_NUM_THREADS = 64
JULIA_EDITOR = code
```
---
### Author:  [@bkamins](https://discourse.julialang.org/u/bkamins)
#### Post date: [April 27, 2021, 10:15pm UTC](https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130/2 "2021-04-27T22:15:04Z")
In `df1` you are already providing concrete types. If you did the following (as you do with a `Matrix`) you would also get `Any` as `eltype`:
```julia
julia> DataFrame(Name = Any["this", "that"],
Radius = Any[1.0, 2.0],
SemiDiameter = Any[4.0, missing])
2×3 DataFrame
Row │ Name Radius SemiDiameter
│ Any Any Any
─────┼────────────────────────────
1 │ this 1.0 4.0
2 │ that 2.0 missing
```
The reason is that now `DataFrame` constructor takes `eltype` of the source to construct columns. I find this behavior natural, as otherwise (if types were narrowed down) you could have problems (I will show them later). First let me show how to narrow down the `eltype`. It is quite easy using the `identity` function and broadcasting it:
```julia
julia> df = DataFrame(Name = Any["this", "that"],
Radius = Any[1.0, 2.0],
SemiDiameter = Any[4.0, missing])
2×3 DataFrame
Row │ Name Radius SemiDiameter
│ Any Any Any
─────┼────────────────────────────
1 │ this 1.0 4.0
2 │ that 2.0 missing
julia> identity.(df)
2×3 DataFrame
Row │ Name Radius SemiDiameter
│ String Float64 Float64?
─────┼───────────────────────────────
1 │ this 1.0 4.0
2 │ that 2.0 missing
```
So now the example of the problem:
```julia
julia> df = DataFrame([1 2; missing 3], :auto)
2×2 DataFrame
Row │ x1 x2
│ Int64? Int64?
─────┼─────────────────
1 │ 1 2
2 │ missing 3
julia> df2 = identity.(df) # narrow down eltype
2×2 DataFrame
Row │ x1 x2
│ Int64? Int64
─────┼────────────────
1 │ 1 2
2 │ missing 3
julia> df2[2, 2] = missing
ERROR: MethodError: Cannot `convert` an object of type Missing to an object of type Int64
```
and you get an error because `eltype` is too narrow although you might have assumed that `missing` will be allowed as your source container allowed it for all columns; doing the same operation on `df` works:
```julia
julia> df[2, 2] = missing
missing
julia> df
2×2 DataFrame
Row │ x1 x2
│ Int64? Int64?
─────┼──────────────────
1 │ 1 2
2 │ missing missing
```
---
### Author:  [@brianguenter](https://discourse.julialang.org/u/brianguenter)
#### Post date: [April 27, 2021, 11:11pm UTC](https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130/3 "2021-04-27T23:11:15Z")
Thank you, that solves the problem.
I have never seen the identity function used in this way and it’s mysterious to me how it works. Could you explain the mechanism?
---
### Author:  [@bkamins](https://discourse.julialang.org/u/bkamins)
#### Post date: [April 28, 2021, 8:35am UTC](https://discourse.julialang.org/t/dataframe-parses-differently-if-data-is-passed-in-columns-vs-as-an-array/60130/4 "2021-04-28T08:35:25Z")
Broadcasted `identity`, comprehensions, and `map` dynamically determine the `eltype` of the resulting container by adjusting it to the result of the transformation:
```julia
julia> x = Any[1, 2, 3]
3-element Vector{Any}:
1
2
3
julia> identity.(x)
3-element Vector{Int64}:
1
2
3
julia> [v for v in x]
3-element Vector{Int64}:
1
2
3
julia> map(identity, x)
3-element Vector{Int64}:
1
2
3
```
A different behavior happens with indexing, which reuses `eltype` of source:
```julia
julia> x[1:2]
2-element Vector{Any}:
1
2
```