# SQLite, DataFrame and Missing

**URL:** https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171
**Category:** Data
**Created:** [November 27, 2021, 3:38pm UTC](https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171 "2021-11-27T15:38:11Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![JackStrauss](https://avatars.discourse-cdn.com/v4/letter/j/f14d63/32.png) [@JackStrauss](https://discourse.julialang.org/u/JackStrauss)
#### Post date: [November 27, 2021, 3:38pm UTC](https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171/1 "2021-11-27T15:38:11Z")

</div>

Hello

Julia is wonderful, but this got me stymied

I am reading a sqlite table into a dataframe, where one column is  
r real  
and all values are missing. I use  
df=DBInterface.execute(db, sql) |\> DataFrame;  
and get df.r to be of type missing (think it should be Union{Missing,Float64} )  
and then I am unable to change any values in df.r as it is missing  
when I try to use the tricks suggested here ([DataFrames: convert column data type - #50 by Skoffer](https://discourse.julialang.org/t/dataframes-convert-column-data-type/35522/50)) to convert the column, it always just gives me missing back, like  
convert.(Union{Missing,Float64},df[!,:r])

I can bypass this by making a new column like Vector{Union{Missing,Float64}}(undef,size(df)[1]) if sum(ismissing.(df.r))==size(df)[1], but that is horribly clumcy.

SQLite.jl returns the correct type, but it fails when converted to a dataframe.  
So, I was wondering if there was a way to make SQLite.jl honor the sql types when read into a dataframe, or convert a dataframe with all missing to a union of missing and float?

all the best, Jack

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [November 27, 2021, 8:43pm UTC](https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171/2 "2021-11-27T20:43:32Z")

</div>

This should be fixable. Though I’m not a maintainer of SQLite.jl, it can probably be fixed there. Please file an issue so the maintainers see this.

---

<div class="post-metadata">

### Author: ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)
#### Post date: [November 27, 2021, 9:54pm UTC](https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171/3 "2021-11-27T21:54:37Z")

</div>

> [@JackStrauss](#):
>
> when I try to use the tricks suggested here ([DataFrames: convert column data type - #50 by Skoffer](https://discourse.julialang.org/t/dataframes-convert-column-data-type/35522/50)) to convert the column, it always just gives me missing back, like  
> convert.(Union{Missing,Float64},df[!,:r])

Do `convert(Vector{Union{Missing, Float64}, df.r)` instead.

But I agree with @pdeffebach that something seems to be wrong in SQLite.jl or DBInterface.jl.

---

<div class="post-metadata">

### Author: ![JackStrauss](https://avatars.discourse-cdn.com/v4/letter/j/f14d63/32.png) [@JackStrauss](https://discourse.julialang.org/u/JackStrauss)
#### Post date: [November 28, 2021, 8:25am UTC](https://discourse.julialang.org/t/sqlite-dataframe-and-missing/72171/4 "2021-11-28T08:25:19Z")

</div>

Fantastic,

thank you both, @pdeffebach I filed an issue in SQlite, and @nalimilan , makes sense. To my untrained eye, both should do the same, but I can appreciate the subtlety in the difference.

best, wishes, Jack.
