# Issues querying a DataFrame

**URL:** <https://discourse.julialang.org/t/issues-querying-a-dataframe/34943>\
**Category:** General Usage\
**Tags:** query, dataframes\
**Created:** [February 20, 2020, 11:05pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943 "2020-02-20T23:05:31Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bensetterholm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bensetterholm/32/12792_2.png) [@bensetterholm](https://discourse.julialang.org/u/bensetterholm)\
**Post date:** [February 20, 2020, 11:05pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/1 "2020-02-20T23:05:31Z")

</div>

Suppose I have a DataFrame `df` which was generated by a Dictionary which may include whitespace in the keys

```julia
using DataFrames

myDict = Dict()
myDict["aKey"] = 1:10
myDict["anotherKey"] = zeros(10)
myDict["a_nice_key"] = rand(Bool, 10)
myDict["a nasty key"] = fill("oof", 10)
df = DataFrame(myDict)

```

If I wanted to select the colomns `a_nice_key` and `anotherKey` where `aKey` is less than say 4, this is easy to do with Query.jl

```julia
using Query

x = @from i in df begin
    @where i.aKey < 4
    @select {i.a_nice_key, i.anotherKey}
    @collect DataFrame
end

```

What I cannot figure out is how to select elements from the `a nasty key` column since its symbol is not simple. For example, this does not work

```julia
x = @from i in df begin
    @where i.aKey < 4
    @select i[!, Symbol("a nasty key")]
    @collect DataFrame
end

```

Is there some way I can work around this? I have tried various permutations but cannot find any way to access these columns in my DataFrame.

I get into the same trouble with the DataFramesMeta package (resetting the REPL to clear the name conflicts with Query.jl)

```julia
using DataFramesMeta, Lazy

x = @> begin
    df
    @where(:aKey .< 4)
    @select(:a_nice_key, Symbol("a nasty key"))
end

```

(Interestingly enough, `select` works with one string turned into a `Symbol`, but no more than one).

Finally, in neither the Query not DataFramesMeta packages can I figure out how to splat a predefined list of column names to select. For example, with DataFramesMeta:

```julia
colsOfInterest = [:anotherKey, :a_nice_key]

x = @> begin
    df
    @where(:aKey .< 4)
    @select(colsOfInterest...)
end

```

fails. Is there a way to resolve both these two issues in either one of these (or another querying) package?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 21, 2020, 12:34am UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/2 "2020-02-21T00:34:49Z")

</div>

Thanks for this, OP.

DataFramesMeta definitely needs some work on this. You should file an issue there.

Sorry for the frustration on this, I understand the benefit of a tidyverse-style string of commands. Hopefully progress will be made on this soon.

---

<div class="post-metadata">

**Author:** ![bensetterholm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bensetterholm/32/12792_2.png) [@bensetterholm](https://discourse.julialang.org/u/bensetterholm)\
**Post date:** [February 21, 2020, 2:37pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/3 "2020-02-21T14:37:46Z")

</div>

Thanks!

With regards to my first problem (symbols with whitespace, or hyphens, or other nasty characters that cannot be represented with a preceding colon), I see that there is already a related unresolved [issue](https://github.com/JuliaData/DataFramesMeta.jl/issues/93) in DataFramesMeta, so I didn’t raise a new issue there. (I will note that the suggested “hack” using a `cols` function seems to be no longer available.)

With regards to my second problem, I found a loosely related [closed issue](https://github.com/JuliaData/DataFramesMeta.jl/issues/87) which inspired me to try the following and fortuitously works for solving both of my problems! I hadn’t considered passing the vector of symbols _without_ splatting.

```julia
julia> using DataFrames, DataFramesMeta, Lazy

julia> df = DataFrame(Dict("aKey" => 1:5,
                           "anotherKey" => zeros(5),
                           "yetAnotherKey" => ones(5),
                           "a nasty key" => fill(2, 5),
                           "another nasty key" => 2:6,))
5×5 DataFrame
│ Row │ a nasty key │ aKey │ another nasty key │ anotherKey │ yetAnotherKey │
│ │ Int64 │ Int64 │ Int64 │ Float64 │ Float64 │
├─────┼─────────────┼───────┼───────────────────┼────────────┼───────────────┤
│ 1 │ 2 │ 1 │ 2 │ 0.0 │ 1.0 │
│ 2 │ 2 │ 2 │ 3 │ 0.0 │ 1.0 │
│ 3 │ 2 │ 3 │ 4 │ 0.0 │ 1.0 │
│ 4 │ 2 │ 4 │ 5 │ 0.0 │ 1.0 │
│ 5 │ 2 │ 5 │ 6 │ 0.0 │ 1.0 │

julia> colsOfInterest = Symbol.(["a nasty key", "another nasty key", "anotherKey"])
3-element Array{Symbol,1}:
 Symbol("a nasty key")      
 Symbol("another nasty key")
 :anotherKey                

julia> x = @> begin
           df
           @where(:aKey .< 4)
           @select(colsOfInterest)
       end
3×3 DataFrame
│ Row │ a nasty key │ another nasty key │ anotherKey │
│ │ Int64 │ Int64 │ Float64 │
├─────┼─────────────┼───────────────────┼────────────┤
│ 1 │ 2 │ 2 │ 0.0 │
│ 2 │ 2 │ 3 │ 0.0 │
│ 3 │ 2 │ 4 │ 0.0 │

```

Since I have a solution that for now _seems_ to work, albeit with different syntax than I naïvely expected, do you still suggest I raise an issue in DataFramesMeta?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 21, 2020, 6:19pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/4 "2020-02-21T18:19:22Z")

</div>

You should still file an issue for sure.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [February 21, 2020, 7:47pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/5 "2020-02-21T19:47:08Z")

</div>

OP do the right thing and abandon symbols with whitespace!

---

<div class="post-metadata">

**Author:** ![bensetterholm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bensetterholm/32/12792_2.png) [@bensetterholm](https://discourse.julialang.org/u/bensetterholm)\
**Post date:** [February 21, 2020, 8:21pm UTC](https://discourse.julialang.org/t/issues-querying-a-dataframe/34943/6 "2020-02-21T20:21:57Z")

</div>

While that _sounds_ like a sensible solution, it isn’t worthwhile in my application to create all the boilerplate necessary to implement non-whitespace-symbols in practice.

DataFrames are a convenient internal data structure for me to reason about my input data (which I have no control over and _may_ include whitespace/hyphens) and to manipulate it without having to build my own bespoke objects and query functions. (Any performance penalties suffered for this abstraction are negligible for this use case.)

Since I was able to get it to work in the end _with_ whitespaces, etc., I am content. I will raise an issue though at the request of @pdeffebach.