Converting array of array of array to multidimensional array

General Usage
1

Hi,

Not sure if this has been answered already.

My question is similar to this topic: Very best way to concatenate an array of arrays

Basically, I want a more general solution. That is, if I have a multidimensional array, I want to convert it into a nested array, and vice versa.

For example, in Python, one could do this:

import numpy as np
x = np.random.randn(3, 2, 4)
x_list = x.tolist()
x == np.array(x_list)  # True

How would I translate this to Julia? Or is there something similar? I essentially want:

x = randn(3, 2, 4)
x_list = tolist(x)  # `tolist` is a function I'm looking for
x == toarray(x_list)  # `toarray` is another function I'm looking for

I’d also be interested in generalizing this to higher-order arrays.

Thanks!

EDIT (Feb 8, 2020):

Thanks for the responses so far! I apologize as my original question was unclear. I’ve added the following snippet to clarify my intent.

I’m aware of libraries like BSON.jl and JLD(2).jl. But I am sometimes interested in saving multidimensional arrays (like x, a 3D array, in my example below) in JSON format for compatibility with other software. The issue is when I “json-ify” multidimensional arrays, (jx below), and then try to parse it, I don’t get back a multidimensional array, but an Array{Any, 1}, which in this case is really a nested array (specifically, array of array of arrays, px below).

I can come up with a hack to convert this back to the original format (x_again), but I am curious if there is a cleaner way to do this. It would also be nice if there’s a solution which generalizes to higher order arrays.

Hope this makes sense.

Thanks!

using JSON


# Generate an Array of dimensions (2, 3, 4)
x = reshape(collect(1:24), 2, 3, 4)

# 2Ă—3Ă—4 Array{Int64,3}:
# [:, :, 1] =
#  1  3  5
#  2  4  6
# 
# [:, :, 2] =
#  7   9  11
#  8  10  12
# 
# [:, :, 3] =
#  13  15  17
#  14  16  18
# 
# [:, :, 4] =
#  19  21  23
#  20  22  24

# Convert to JSON
jx = JSON.json(x)

# "[[[1,2],[3,4],[5,6]],[[7,8],[9,10],[11,12]],[[13,14],[15,16],[17,18]],[[19,20],[21,22],[23,24]]]"

# Parse JSON
px = JSON.parse(jx)

# 4-element Array{Any,1}:
#  Any[Any[1, 2], Any[3, 4], Any[5, 6]]
#  Any[Any[7, 8], Any[9, 10], Any[11, 12]]
#  Any[Any[13, 14], Any[15, 16], Any[17, 18]]
#  Any[Any[19, 20], Any[21, 22], Any[23, 24]]

# Convert parsed JSON back to original array, x
x_again = [px[i][j][k]
           for k in 1:length(px[1][1]),
           j in 1:length(px[1]),
           i in 1:length(px)]

# 2Ă—3Ă—4 Array{Int64,3}:
# [:, :, 1] =
#  1  3  5
#  2  4  6
# 
# [:, :, 2] =
#  7   9  11
#  8  10  12
# 
# [:, :, 3] =
#  13  15  17
#  14  16  18
# 
# [:, :, 4] =
#  19  21  23
#  20  22  24

# Assert all elements are the same
@assert all(x .== x_again)  # true
2

First of all, welcome to Julia Discourse!

I’m not sure if I understand this correctly, but the result of the function to_list is an object that has an indexing of the the form a[1][2][3] when the corresponding indexing access of the original array its a[1,2,3],is that right?

3

You may be looking for

1 Like
4

Thanks for the reply! This is close to what I am looking for. As I wasn’t clear before, I’ve updated my original question.

5

Thanks! That is correct. I’ve updated my question for clarity.

6

I just ran into this as I imported some json exported from python’s arrays. Your method works.

7

Hello,
I do have a similar issue. I want to convert Vector{Vector{Vector{Float64}}} efficiently to Array{Float64,3} and have been lost for the past 48 hours. I would appreciate any helps on the matter.

8

Me, it has been for the past 2 years…

The simplest syntax is probably provided by TensorCast:

using TensorCast    
@cast w[i,j,k] |= v[i][j][k]  # greedy
@cast w[i,j,k] := v[i][j][k]  # lazy
9

SplitApplyCombine is 2x faster, which seems pretty common:

julia> A = [[rand(100) for _ in 1:100] for _ in 1:100];

julia> using SplitApplyCombine
julia> @btime combinedims(combinedims($A));
  1.636 ms (6 allocations: 7.71 MiB)

julia> using TensorCast   
julia> @btime @cast w[i,j,k] |= $A[i][j][k];
  3.525 ms (7 allocations: 7.63 MiB)
10

The SplitApplyCombine result is transposed and to compute that, it takes also half the time in TensorCast:

@cast w2[i,j,k] |= A[k][j][i]

The advantage of TensorCast is that the syntax is crystal clear on what goes where and how, with no mental burden on the user. I wish the Julia language would include such a syntax by default.

2 Likes