How to convert Vector of Vectors to Matrix

I have a vector = [[1,0,1],[ 0,0,1]]

I am trying to convert this to → [ 1 0 1 ; 0 0 1 ]

How?

2 Likes

What have you tried so far?

The answer is

julia> mapreduce(permutedims, vcat, x)
2×3 Matrix{Int64}:
 1  0  1
 0  0  1
10 Likes

This does the job too:

permutedims(hcat(x...))
3 Likes

This could be a bit faster.

julia> reduce(vcat,transpose.(x))
2×3 Matrix{Int64}:
 1  0  1
 0  0  1
5 Likes

This is not desirable if x has many elements. But for the two-element array in the original example, it’s fine.

For a large number of elements could do instead:

reduce(hcat,x)'
4 Likes

I noticed that I ran into a problem when there were more than 10,000 elements… Is there a workaround for this limitation? (Code would fail if I had 10,001 vector of vectors)

Can you post an MWE? This is unexpected

1 Like

I will; I am working on a notebook right now that once tidied up, I will send a link show a working example…
-p

Please don’t send a link to a notebook. See if you can write the smallest amount of code that reproduces the problem and copy and paste it onto discourse.

1 Like

will do

Don’t use splatting (...) of large arrays, e.g. don’t use things like hcat(x...). Use reduce, e.g. reduce(hcat,x) as suggested above.

6 Likes

I made this function in a package. Don’t know if it is better or worse than reduce(hcat, x)

function vector_of_vector_to_matrix(vov::Vector{Vector{T}}) where T
    n = length(vov)
    p = length(vov[1])
    mat = Matrix{T}(undef, n, p)
    for j in 1:p, i in 1:n
        mat[i, j] = vov[i][j]
    end
    return mat
end

There are already packages with such a function: see combinedims() in SplitApplyCombine.jl. It also has a view-based version that doesn’t copy data: combinedimsview().

1 Like

Check it with BenchmarkTools.
For, example:

vov = [rand(0:1,100) for _ in 1:100_000]
using BenchmarkTools
@btime vector_of_vector_to_matrix($vov)     # 160 ms (2 allocations: 76 MiB)
@btime reduce(hcat,$vov)'                   #  17 ms (2 allocations: 76 MiB)
2 Likes

You don’t materialize the transpose in the latter case though. Should probably be included in the benchmark.

I run into the same problem by getting a vector of vector datatype with [rand(x) for i=1:AM_SAMPLES], where rand(x) gives a vector. Afterward, I want a matrix out of it, but maybe I could avoid it beforehand. I am looking forward to your suggestions.

for me hcat(x...) did the job. I am wondering how hcat(x...) actually works

I am wondering how hcat(x...) actually works

hcat() concatenates the arrays given as arguments along dimension 2. In your example when using the splatting operator ... it does the same as writing explicitely the different element vectors of x:

x = [rand(10) for _ in 1:3]             # 3-element Vector{Vector{Float64}}
hcat(x...) == hcat(x[1], x[2], x[3])    # true

As indicated, this is not the most efficient way of doing it when there are many elements to splat. The alternative: reduce(hcat,x) should be much better.

@rafael.guerra thank you for that example. I see that it works, but still, I am a bit confused about the splitting operator … Here is the example I have used:

julia> x=[1., 2., 3.]
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> @show x...
ERROR: syntax: "..." expression outside call around ..
Stacktrace:
 [1] top-level scope
   @ none:1

julia> @show [x...]
[x...] = [1.0, 2.0, 3.0]
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> hcat(x...)
1×3 Matrix{Float64}:
 1.0  2.0  3.0

julia> hcat(x)
3×1 Matrix{Float64}:
 1.0
 2.0
 3.0

In this example, I see that I need brackets for the splatting operator. [x…] is still a 3 element vector in this example, but Julia does make a difference when the splatting operator is used. The splatting command allows julia to shred the vector in each element.