How to convert Vector of Vectors to Matrix

General Usage
,
1

I have a vector = [[1,0,1],[ 0,0,1]]

I am trying to convert this to → [ 1 0 1 ; 0 0 1 ]

How?

5 Likes
2

What have you tried so far?

The answer is

julia> mapreduce(permutedims, vcat, x)
2×3 Matrix{Int64}:
 1  0  1
 0  0  1
13 Likes
3

This does the job too:

permutedims(hcat(x...))
5 Likes
4

This could be a bit faster.

julia> reduce(vcat,transpose.(x))
2×3 Matrix{Int64}:
 1  0  1
 0  0  1
8 Likes
5

This is not desirable if x has many elements. But for the two-element array in the original example, it’s fine.

6

For a large number of elements could do instead:

reduce(hcat,x)'
8 Likes
7

I noticed that I ran into a problem when there were more than 10,000 elements… Is there a workaround for this limitation? (Code would fail if I had 10,001 vector of vectors)

1 Like
8

Can you post an MWE? This is unexpected

1 Like
9

I will; I am working on a notebook right now that once tidied up, I will send a link show a working example…
-p

10

Please don’t send a link to a notebook. See if you can write the smallest amount of code that reproduces the problem and copy and paste it onto discourse.

2 Likes
11

will do

12

Don’t use splatting (...) of large arrays, e.g. don’t use things like hcat(x...). Use reduce, e.g. reduce(hcat,x) as suggested above.

Update: In Julia 1.9 or later, use the stack function, e.g. stack([[1,0,1],[ 0,0,1]], dims=1) == [ 1 0 1 ; 0 0 1 ].

19 Likes
13

I made this function in a package. Don’t know if it is better or worse than reduce(hcat, x)

function vector_of_vector_to_matrix(vov::Vector{Vector{T}}) where T
    n = length(vov)
    p = length(vov[1])
    mat = Matrix{T}(undef, n, p)
    for j in 1:p, i in 1:n
        mat[i, j] = vov[i][j]
    end
    return mat
end
1 Like
14

There are already packages with such a function: see combinedims() in SplitApplyCombine.jl. It also has a view-based version that doesn’t copy data: combinedimsview().

1 Like
15

Check it with BenchmarkTools.
For, example:

vov = [rand(0:1,100) for _ in 1:100_000]
using BenchmarkTools
@btime vector_of_vector_to_matrix($vov)     # 160 ms (2 allocations: 76 MiB)
@btime reduce(hcat,$vov)'                   #  17 ms (2 allocations: 76 MiB)
2 Likes
16

You don’t materialize the transpose in the latter case though. Should probably be included in the benchmark.

17

I run into the same problem by getting a vector of vector datatype with [rand(x) for i=1:AM_SAMPLES], where rand(x) gives a vector. Afterward, I want a matrix out of it, but maybe I could avoid it beforehand. I am looking forward to your suggestions.

for me hcat(x...) did the job. I am wondering how hcat(x...) actually works

1 Like
18

I am wondering how hcat(x...) actually works

19

hcat() concatenates the arrays given as arguments along dimension 2. In your example when using the splatting operator ... it does the same as writing explicitely the different element vectors of x:

x = [rand(10) for _ in 1:3]             # 3-element Vector{Vector{Float64}}
hcat(x...) == hcat(x[1], x[2], x[3])    # true

As indicated, this is not the most efficient way of doing it when there are many elements to splat. The alternative: reduce(hcat,x) should be much better.

20

@rafael.guerra thank you for that example. I see that it works, but still, I am a bit confused about the splitting operator … Here is the example I have used:

julia> x=[1., 2., 3.]
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> @show x...
ERROR: syntax: "..." expression outside call around ..
Stacktrace:
 [1] top-level scope
   @ none:1

julia> @show [x...]
[x...] = [1.0, 2.0, 3.0]
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> hcat(x...)
1×3 Matrix{Float64}:
 1.0  2.0  3.0

julia> hcat(x)
3×1 Matrix{Float64}:
 1.0
 2.0
 3.0

In this example, I see that I need brackets for the splatting operator. [x…] is still a 3 element vector in this example, but Julia does make a difference when the splatting operator is used. The splatting command allows julia to shred the vector in each element.