How to efficiently re-arrange a vector of vectors into a matrix?

New to Julia
1

Hi! I would like to turn a vector of vectors into a matrix where each column corresponds to one of the input vectors. This could be accomplished with for instance the following:

julia> a = [[1, 2], [3, 4], [5, 6]]
3-element Vector{Vector{Int64}}:
 [1, 2]
 [3, 4]
 [5, 6]

julia> reshape(collect(Iterators.flatten(a)), (length(a[1]),length(a)))
2×3 Matrix{Int64}:
 1  3  5
 2  4  6

Is it possible to do this operation more efficiently? I wonder if it is possible to create a @view object to avoid allocations.

julia> using BenchmarkTools

julia> @btime reshape(collect(Iterators.flatten($a)), (length($a[1]),length($a)))
  141.132 ns (5 allocations: 304 bytes)
2

The way I have always seen it done is
hcat(a...)
Which brought the allocations from 7 to 2, for me, and sped it up by 2×.

3

SplitApplyCombine.jl has some utilities for this kind of thing, combinedims: https://github.com/JuliaData/SplitApplyCombine.jl#combinedimsarray (and a lazy view version combinedimsview as well).

4

It’s generally faster to do reduce(hcat, a) since reduce has a special method for hcat and vcat which avoids some unnecessary allocations:

julia> a = [rand(2) for _ in 1:1000];

julia> using BenchmarkTools

julia> @btime reduce(hcat, $a);
  5.456 μs (1 allocation: 15.75 KiB)

julia> @btime hcat($a...);
  19.715 μs (6 allocations: 47.42 KiB)

If you actually have a bunch of small vectors with the same size, then it’s even faster (basically free) to use reinterpret with StaticArrays:

julia> using StaticArrays: SVector

julia> a = [rand(SVector{2, Float64}) for _ in 1:1000];

julia> @btime reshape(reinterpret(Float64, $a), (2, :));
  10.507 ns (0 allocations: 0 bytes)

That’s 10 nanoseconds and no allocation compared to 5 microseconds and 1 allocation for the reduce(hcat, a) version.

9 Likes
5

For the record, using TensorCast nice syntax:

using TensorCast
@cast b[j,i] := v[i][j]   # v is vector of vectors
@cast b[j,i] := sv[i]{j}  # sv is vector of SVectors
4 Likes
6 
julia> a = [[1, 2], [3, 4], [5, 6]]
julia> reduce(hcat, a)
2×3 Matrix{Int64}:
 1  3  5
 2  4  6
julia> reduce(vcat, a')
3×2 Matrix{Int64}:
 1  2
 3  4
 5  6
julia> [a...;;]
2×3 Matrix{Int64}:
 1  3  5
 2  4  6
julia> [a'...;]
3×2 Matrix{Int64}:
 1  2
 3  4
 5  6

Many flavors, although even if i like Julia, i really think there’s a bag of tricks style in some points…

3 Likes