# Query.jl: Selection using Base functions and possibly missing values

**URL:** https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557
**Category:** Data
**Tags:** query
**Created:** [June 30, 2017, 4:17am UTC](https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557 "2017-06-30T04:17:22Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![tcovert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tcovert/32/3855_2.png) [@tcovert](https://discourse.julialang.org/u/tcovert)
#### Post date: [June 30, 2017, 4:17am UTC](https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557/1 "2017-06-30T04:17:22Z")

</div>

Suppose I have a DataFrame with two fields: `idx` and `date`. The `date` field has missing values (in the `DataFrames` sense) and is currently stored in the DataFrame as a string. Is there a query statement that I can write which parses the string into a date? I tried something like this:

```julia
df2 = @from i in df begin
       @select {i.idx, date = Date.(i.date, "mm/dd/yyyy")}
       @collect DataFrame
       end

```

but got an error like this:

```julia
ERROR: type UnionAll has no field parameters
Stacktrace:
 [1] column_types at /Users/tcovert/.julia/v0.6/IterableTables/src/utilities.jl:20 [inlined]
 [2] _DataFrame(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},_} where _,Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##11#13}) at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:105
 [3] DataFrames.DataFrame(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},_} where _,Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##11#13}) at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:128
 [4] collect(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},_} where _,Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##11#13}, ::Type{DataFrames.DataFrame}) at /Users/tcovert/.julia/v0.6/Query/src/sinks/sink_type.jl:2

```

I also tried a version with no dot-broadcasting:

```julia
df2 = @from i in df begin
       @select {i.idx, date = Date(i.date, "mm/dd/yyyy")}
       @collect DataFrame
       end

```

and got this error:

```julia
ERROR: MethodError: Cannot `convert` an object of type DataValues.DataValue{String} to an object of type Int64
This may have arisen from a call to the constructor Int64(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] next at /Users/tcovert/.julia/v0.6/Query/src/enumerable/enumerable_select.jl:41 [inlined]
 [2] macro expansion at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:91 [inlined]
 [3] _filldf(::Tuple{DataArrays.DataArray{Int64,1},Array{Date,1}}, ::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},Date},Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##15#16}) at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:79
 [4] _DataFrame(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},Date},Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##15#16}) at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:119
 [5] DataFrames.DataFrame(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},Date},Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##15#16}) at /Users/tcovert/.julia/v0.6/IterableTables/src/integrations/dataframes.jl:128
 [6] collect(::Query.EnumerableSelect{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},Date},Query.EnumerableIterable{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},IterableTables.DataFrameIterator{NamedTuples._NT_idx_date{DataValues.DataValue{Int64},DataValues.DataValue{String}},Tuple{DataArrays.DataArray{Int64,1},DataArrays.DataArray{String,1}}}},##15#16}, ::Type{DataFrames.DataFrame}) at /Users/tcovert/.julia/v0.6/Query/src/sinks/sink_type.jl:2

```

is what I am trying to do possible? if so, what am I doing wrong?

thanks in advance for any suggestions you can offer.

here is some example data to apply the code to above: [Dropbox - query\_example.csv - Simplify your life](https://www.dropbox.com/s/kgiicawhegmtavc/query_example.csv?dl=0)

(also posted here: [https://github.com/davidanthoff/Query.jl/issues/134](https://github.com/davidanthoff/Query.jl/issues/134))

---

<div class="post-metadata">

### Author: ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)
#### Post date: [June 30, 2017, 5:34am UTC](https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557/2 "2017-06-30T05:34:51Z")

</div>

Please do not double post on an issue tracker and discourse. It seems to me that @davidanthoff already pointed you towards a solution.

[https://github.com/davidanthoff/Query.jl/issues/134#issuecomment-312176898](https://github.com/davidanthoff/Query.jl/issues/134#issuecomment-312176898)

---

<div class="post-metadata">

### Author: ![tcovert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tcovert/32/3855_2.png) [@tcovert](https://discourse.julialang.org/u/tcovert)
#### Post date: [June 30, 2017, 5:42am UTC](https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557/3 "2017-06-30T05:42:39Z")

</div>

I guess I am still confused about where to post questions about things that appear to be legitimate bugs. There is a clear message in the `Query` issue tracker that suggests questions about usage should go here, but it wasn’t obvious to me whether I would be more likely to get an answer here or there. In the future, where should questions like this be sent?

---

<div class="post-metadata">

### Author: ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [June 30, 2017, 7:16am UTC](https://discourse.julialang.org/t/query-jl-selection-using-base-functions-and-possibly-missing-values/4557/4 "2017-06-30T07:16:51Z")

</div>

I monitor both the github repo and the forum here, so I’ll see posts on either site. I would prefer to have usage questions here, and if it turns out that a usage question actually uncovered a bug in Query.jl, I’ll open an issue on github. If it is a clear bug, start with an issue. When in doubt, start here, we can always open an issue later.
