# Can DataFrames.jl handle multi-dimensional arrays

**URL:** <https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727>\
**Category:** General Usage\
**Tags:** question\
**Created:** [November 5, 2019, 7:06am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727 "2019-11-05T07:06:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![00krishna](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00krishna/32/8843_2.png) [@00krishna](https://discourse.julialang.org/u/00krishna)\
**Post date:** [November 5, 2019, 7:06am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/1 "2019-11-05T07:06:42Z")

</div>

I was wondering if `DataFrames.jl` can handle multi-dimensional arrays, meaning an array with 3 or more dimensions.

I can run

```julia
using DataFrames
DataFrame(rand(3,4)

```

and that works fine.

But if I try:

```julia
DataFrame(rand(3,4,5))

```

Then I get a rather cryptic error message  
`ERROR: ArgumentError: 'Array{Float64,3}' iterates 'Float64' values, which don't satisfy the Tables.jl Row-iterator interface`

Does anyone know if there is a plan to add multi-dimensional support to the package. I checked the issue list in github but did not see any issues that explicitly mentioned this–but I did not dig too deeply. So I thought I would ask. Thanks.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [November 5, 2019, 7:57am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/2 "2019-11-05T07:57:30Z")

</div>

Currently DataFrames.jl support only data that can be represented as a two-dimensional object (i.e. a list of vectors). There are no plans currently to change this.

However, note that you can easily store any data type in a cell of a `DataFrame`, so e.g. you can write the following:

```julia
julia> DataFrame([rand(2) for _ in 1:4, _ in 1:3])
4×3 DataFrame
│ Row │ x1 │ x2 │ x3 │
│ │ Array{Float64,1} │ Array{Float64,1} │ Array{Float64,1} │
├─────┼───────────────────────┼───────────────────────┼────────────────────────┤
│ 1 │ [0.382557, 0.0063816] │ [0.367795, 0.301294] │ [0.083309, 0.465583] │
│ 2 │ [0.424023, 0.244837] │ [0.233543, 0.834364] │ [0.00453236, 0.548186] │
│ 3 │ [0.392719, 0.628895] │ [0.979969, 0.534259] │ [0.588646, 0.825887] │
│ 4 │ [0.980133, 0.495353] │ [0.291205, 0.0895148] │ [0.535076, 0.982956] │

```

where effectively you have a third dimension nested as a cell value of a two dimensional `DataFrame`.

---

<div class="post-metadata">

**Author:** ![Mattriks](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattriks/32/351_2.png) [@Mattriks](https://discourse.julialang.org/u/Mattriks)\
**Post date:** [November 5, 2019, 8:23am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/3 "2019-11-05T08:23:38Z")

</div>

or

```julia
X = rand(3,4,5)
df = DataFrame(X=collect(eachslice(X, dims=3)))

```

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [November 5, 2019, 8:23am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/4 "2019-11-05T08:23:48Z")

</div>

You may also be looking for things like [AxisArrays](https://github.com/JuliaArrays/AxisArrays.jl), which have some similarities but allow any number of dimensions.

See also [NamedDims](https://github.com/invenia/NamedDims.jl) and a small zoo of other recent packages discussed [here](https://github.com/JuliaCollections/AxisArraysFuture/issues/1).

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [November 5, 2019, 8:26am UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/5 "2019-11-05T08:26:55Z")

</div>

I work a lot with panel data, for which a “3D” representation often seems natural at first. In this context I always find it interesting that Python’s pandas is named after **pan** el **da** ta, and indeed used to have a `Panel` object in addition to the standard `DataFrame`, but this has been deprecated for a while now in favour of multi-indices on a regular DataFrame: [pandas.Panel — pandas 0.23.4 documentation](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.Panel.html)

---

<div class="post-metadata">

**Author:** ![00krishna](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/00krishna/32/8843_2.png) [@00krishna](https://discourse.julialang.org/u/00krishna)\
**Post date:** [November 6, 2019, 8:01pm UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/6 "2019-11-06T20:01:35Z")

</div>

@nilshg this multi-indexing idea is interesting, but seems rather confusing. Like I get the basic idea where instead of having a 3-D volume, you can basically flatten the 3-D into a 2-D array but have a multi-index to recover or iterate over that 3-D structure. So if I run a model for 100 runs, and each model has a duration of 10 years. Then I would normally have a volume of [year, parameters, run]. So each run would have a 10 rows by n number of columns, and then the depth dimension would reference the run number. But I could convert this to a multi-index where I have a 2-D array with a “year” and a “run” column. So the (run, year) would be the multi-index.

But is there a good explanation of how to use multi-indices in a Julia dataframe. The python pandas documentation was never very clear about this, as far as I remember. Or I always ran into issues with some implicit conversions that turned things into multi-indices and I had to get them out of the multi-index. So if there is a good explanation of this please pass along the link. I can check out the Julia dataframe docs in the mean time.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [November 6, 2019, 8:56pm UTC](https://discourse.julialang.org/t/can-dataframes-jl-handle-multi-dimensional-arrays/30727/7 "2019-11-06T20:56:58Z")

</div>

> But is there a good explanation of how to use multi-indices in a Julia dataframe.

Julia DataFrames don’t have indexes, you can do `by` and `groupby` on any of them, and get the same effect.  
which actually makes it much clearer than Pandas, but slower potentially if you do the same grouping again and again.  
(used to be multindexes were super complex because indexes are like columns but not)