# Natural column order for DataFrames

**URL:** <https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095>\
**Category:** New to Julia\
**Created:** [January 1, 2018, 6:25pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095 "2018-01-01T18:25:45Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 1, 2018, 6:25pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/1 "2018-01-01T18:25:45Z")

</div>

Happy New Year! 🎆🎊🎉

I have a Dict{Symbol,Vector} and want to create a DataFrame out of it. It’s easy enough to do DataFrame(dict) but I also want the columns to appear in the right order. For example,

```julia
julia> x = Dict(:data => Dict(:foo => [1,2,3], :bar => [4,5,6], :mouse => [7,8,9]),
                :symbols => [:foo, :bar, :mouse])
Dict{Symbol,Any} with 2 entries:
  :symbols => Symbol[:foo, :bar, :mouse]
  :data => Dict(:mouse=>[7, 8, 9],:bar=>[4, 5, 6],:foo=>[1, 2, 3])

julia> df = DataFrame(x[:data])
3×3 DataFrames.DataFrame
│ Row │ bar │ foo │ mouse │
├─────┼─────┼─────┼───────┤
│ 1 │ 4 │ 1 │ 7 │
│ 2 │ 5 │ 2 │ 8 │
│ 3 │ 6 │ 3 │ 9 │

```

Since I have the proper order in `x[:symbols]`, I could rearrange it as below. It’s a bit ugly so I’m wondering if there’s a better way… Perhaps DataFrames.jl should have another constructor.

```julia
julia> df = DataFrame([x[:data][v] for v in x[:symbols]], x[:symbols])
3×3 DataFrames.DataFrame
│ Row │ foo │ bar │ mouse │
├─────┼─────┼─────┼───────┤
│ 1 │ 1 │ 4 │ 7 │
│ 2 │ 2 │ 5 │ 8 │
│ 3 │ 3 │ 6 │ 9 │

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 1, 2018, 6:50pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/2 "2018-01-01T18:50:36Z")

</div>

I think that the `Pair{Symbol,Vector}...` constructor should do what you want, ie

```julia
using DataFrames
cols = Dict(:foo => [1,2,3], :bar => [4,5,6], :mouse => [7,8,9])
order = [:foo, :bar, :mouse]
DataFrame([name => cols[name] for name in order]...)

```

otherwise the `DataFrames(columns, cnames)` constructor you are using is fine.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 1, 2018, 9:38pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/3 "2018-01-01T21:38:57Z")

</div>

Thanks @Tamas_Papp . I haven’t thought about that one.

I still hope that DataFrame would have another constructor that just takes a Dict and an array of keys of the specified order. Perhaps I’ll submit a PR there 🙂

Ideally (as in your example):

```julia
DataFrame(cols, order)

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 2, 2018, 6:02am UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/4 "2018-01-02T06:02:33Z")

</div>

When conversions are relatively easy, I don’t think it is a good idea to provide constructors for cases like this (there are many similar ones).

All the constructor would do is apply the same one-liners as above, with the cost of increasing code complexity and maintenance burden for the package.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [January 2, 2018, 10:44am UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/5 "2018-01-02T10:44:46Z")

</div>

Agreed, especially that in this case it can be written in an even simpler form:

```
DataFrame((v => x[:data][v] for v in x[:symbols])...)

```

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [January 2, 2018, 3:45pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/6 "2018-01-02T15:45:59Z")

</div>

Looks like what is really needed here is an ordered dictionary type which would preserve the order in which you want the keys to appear. Adding `DataFrame` constructors isn’t really the appropriate solution.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 2, 2018, 6:49pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/7 "2018-01-02T18:49:05Z")

</div>

@Tamas_Papp and @bkamins, I’m not picking on you guys but I humbly disagree. As a package writer, I believe the interface needs to be as user friendly as possible. Am I the only one who has this need? If everyone like me is writing the same one-liner then it would make more sense put that one line in the package.

I like @nalimilan’s suggestion, however. If we have have a keys() function that takes an OrderedDict object and returns the _natural_ order of the keys, then we can bypass the auto-sorting feature in DataFrames. Hence, we don’t need to burden DataFrames package with the additional constructor.

For reference, DataFrames.jl’s [code](https://github.com/JuliaData/DataFrames.jl/blame/master/src/dataframe/dataframe.jl) look like this: :

```julia
function Base.convert(::Type{DataFrame}, d::Associative)
    colnames = keys(d)
    if isa(d, Dict)
        colnames = sort!(collect(keys(d)))
    else
        colnames = keys(d)
    end
    colindex = Index(Symbol[k for k in colnames])
    columns = Any[d[c] for c in colnames]
    DataFrame(columns, colindex)
end

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 2, 2018, 6:54pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/8 "2018-01-02T18:54:33Z")

</div>

> [@tk3369](#):
>
> If we have have a keys() function that takes an OrderedDict object and returns the natural order of the keys, then we can bypass the auto-sorting feature in DataFrames.

Precisely:

```julia
using DataStructures
cols = OrderedDict(:foo => [1,2,3], :bar => [4,5,6], :mouse => [7,8,9])
DataFrame(cols)

```

does the right thing, since it goes through the second branch.

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [January 2, 2018, 6:54pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/9 "2018-01-02T18:54:58Z")

</div>

I agree that in cases like unambigous conversions such as

```julia
DataFrame(dict::OrderedDict)

```

it would be nice to provide constructors (although it does raise an dependency issue).

However, as a DataFrames user I mostly agree with @Tamas_Papp and @bkamins. I don’t see a need to try to guess every possible use of a constructor when a simple and efficient bit of code like a list comprehension would suffice. After all, one of the primary reasons to use a language like Julia as opposed to a language like C++ is that there’s a lot of code which it’s very simple to write so we don’t require anywhere near as many specialized functions.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [January 2, 2018, 7:01pm UTC](https://discourse.julialang.org/t/natural-column-order-for-dataframes/8095/10 "2018-01-02T19:01:00Z")

</div>

> [@ExpandingMan](#):
>
> it would be nice to provide constructors (although it does raise an dependency issue).

The point is, we don’t need any special constructors. We can just rely on the fact that `keys(::OrderedDict)` returns keys in the right order.