Converting Tables into arrays

Hi, I am converting some columns from a table in to a 5x5 array. I have managed to do it in inefficient way as I first convert the table into a DataFrame and then convert that into an array.

Is there a more efficient way of doing this (that would produce the equivalent of ArrayX shown below)?

I am using the using JuliaDB.jl package, but I think the approach will be similar across different packages. I have provided a MWE below:

using JuliaDB, JuliaDBMeta, DataFrames
DataT = table([1,1,1,1,1,2,2,2], [1,2,3,4,5,1,2,3], [10,15,100,115,101,10,15,100],
[210,215,1200,1215,101,210,215,1200],[310,415,3100,1315,101,310,415,3100],
[410,15,4100,4115,1401,410,15,410],[510,515,1500,5115,5101,510,515,1500];
names = [:Scenario, :ID, :T1, :T2, :T3, :T4, :T5])
# To create one array containing all entries from T1 to T5 (when scenario is 1), involve following steps
# 1. Create a table to read all data in columns T1 to T5, where Scenario is 1
X = table(select(filter(i -> (i.Scenario == 1), DataT), All(Between(:T1, :T5))))
# 2. Convert table X into a DataFrame
df = DataFrame(X)
# 3. Convert the DataFrame into an array
ArrayX = Matrix(df)

This gives the required result

julia> ArrayX
5×5 Array{Int64,2}:
  10   210   310   410   510
  15   215   415    15   515
 100  1200  3100  4100  1500
 115  1215  1315  4115  5115
 101   101   101  1401  5101

X is shown below

Table with 5 rows, 5 columns:
T1   T2    T3    T4    T5
───────────────────────────
10   210   310   410   510
15   215   415   15    515
100  1200  3100  4100  1500
115  1215  1315  4115  5115
101  101   101   1401  5101

I have tried to use collect(X) so as to avoid the intermediate steps, but it creates
the following array of Tuples rather than equivalent of ArrayX (shown above)

5-element Array{NamedTuple{(:T1, :T2, :T3, :T4, :T5),NTuple{5,Int64}},1}:
 (T1 = 10, T2 = 210, T3 = 310, T4 = 410, T5 = 510)
 (T1 = 15, T2 = 215, T3 = 415, T4 = 15, T5 = 515)
 (T1 = 100, T2 = 1200, T3 = 3100, T4 = 4100, T5 = 1500)
 (T1 = 115, T2 = 1215, T3 = 1315, T4 = 4115, T5 = 5115)
 (T1 = 101, T2 = 101, T3 = 101, T4 = 1401, T5 = 5101)
1 Like

I think you want Tables.matrix from Tables.jl, documented here.

2 Likes

Thank you. Tables.matrix seems to be more efficient then my workaround. I will wait to see if someone has any suggestions for how to do this within the JuliaDB package i.e without using Tables.jl.