Nested Vectors/Arrays to multidimensional tensor

General Usage
1

I searched far and wide in the Julia forum and stackoverflow but haven’t found a general answer to my question (or I missed the point of the answers):
How can I collect an arbitrary number of nested vectors/arrays into a single multi dimensional tensor?

A minimum working example that generates a similar set of data is the following
data = [ [rand(13,14) for i in 1:12] for j in 1:11]
data representation: Vector{Array{Array{Float64,2},1}

How can I transform this into a single 4 dimensional tensor of the shape (11, 12, 13, 14)?

Obviously I could write basically boiler plate code on my own but I was wondering whether there is some function/package that does that for me.

The application is computational statistics where I want to manipulate the data tensors along its dimensions directly thus circumventing onerous for-loops.

Thanks for any help in advance! =)

2

Try something like

data = [ [rand(13,14) for i in 1:12] for j in 1:11]
A = permutedims(reshape(mapreduce(xs -> vec(reduce(hcat, xs)), hcat, data),
                        13, 14, 12, :),
                (4, 3, 1, 2))
data[1][1] == A[1, 1, :, :] # true
1 Like
3

I have some packages which may interest you (one just announced this week):

julia> data = [ [rand(13,14) for i in 1:12] for j in 1:11];

julia> using LazyStack 

julia> stack(stack.(data)) |> size # this is a view, copy for Array
(13, 14, 12, 11)

julia> permutedims(stack(stack.(data)), (4,3,1,2)) |> size
(11, 12, 13, 14)

julia> using TensorCast

julia> size(@cast out[a,b,c,d] := data[a][b][c,d])
(11, 12, 13, 14)
2 Likes
4

Thank you for your replies.

I will have a look into the answers. =)

5

Ultimately, I rewrote the methods and worked explicitly with multi-dimensional arrays and cat(array1, array2) and from the start instead of nested arrays.

1 Like
6

I find https://github.com/JuliaData/SplitApplyCombine.jl really useful for this kind of problems, and routinely use its functions. There are both copying and non-copying (i.e. view) versions.

1 Like