# How can I do list comprehension for vcat?

I have many 1-d lists that I want to concatenate to have a 1-d list, probably in a list comprehension style. Following is how I can achieve using explicit for-loop

``````f = []
a = [1,2,3,4,6]
b = [3,4,5,20]
c = [8,4,5,20]
d = [8,1,90,40]
x=[a,b,c,d,a]
for a in x
f= [f;a]
end
f  # I get 22-element Vector which is what I am looking for.
``````

``````julia> f = vcat(a,b,c,d,a)
22-element Vector{Int64}:
1
2
3
4
6
3
4
5
20
⋮
1
90
40
1
2
3
4
6
``````
3 Likes

or, use `reduce(vcat, ...)` if you want to work on the vector-of-vectors `x`:

``````a = [1,2,3,4,6]
b = [3,4,5,20]
c = [8,4,5,20]
d = [8,1,90,40]
x = [a,b,c,d,a]
``````
``````julia> f = reduce(vcat, x)
22-element Vector{Int64}:
1
2
3
⋮
3
4
6
``````
6 Likes

if you don’t necessarily have to use vcat

``````append!(f, x...)
``````

instead if you want to use vcat

``````vcat(x...)
``````
``````[a;b;c;d;a]
``````
1 Like

Splatting can have quite unpredictable performance. In general, `reduce(vcat, x)`

5 Likes

How could we write our own Julia loop competitively?
An attempt:

``````function vcat2(x::AbstractVector{<:AbstractVector})
y = Array{eltype(x[begin])}(undef,sum(length,x))
i = 1
for u in x
l = length(u)
y[i:(i+l-1)] .= u
i += l
end
return y
end

# Ex.1:
a, b, c, d = [1,2,3,4,6], [3,4,5,20], [8,4,5,20], [8,1,90,40]
x = [a,b,c,d,a]
@btime reduce(vcat, \$x)                     #  80 ns (2 allocs: 336 bytes)
@btime vcat(\$x...)                          #  92 ns (1 alloc:  240 bytes)
@btime vcat2(\$x)                            #  85 ns (2 allocs: 336 bytes)

# Ex.2:
using Random
Random.seed!(123)
x = [rand(100) for _ in 1:100]
@btime reduce(vcat, \$x)                     # 5.620 μs (3 allocs: 79 KiB)
@btime vcat(\$x...)                          # 5.740 μs (2 allocs: 78 KiB)
@btime vcat2(\$x)                            # 5.560 μs (2 allocs: 78 KiB)

``````
1 Like
``````julia> @btime vcat(x...);
79.442 ns (1 allocation: 240 bytes)

julia> @btime reduce(vcat, x)  ;
83.731 ns (2 allocations: 336 bytes)

julia> @btime vcat2(x) ;
72.587 ns (2 allocations: 336 bytes)

``````
1 Like

Thank you very much for the lovely answer. Out of all the good answers, I used

reduce(vcat, x)

and it worked great until I had empty x. Empty x gave me the following error

``````ArgumentError: reducing over an empty collection is not allowed
``````

Is there any way to resolve this issue? Thanks for the response.

You need to specify an `init`

``````julia> reduce(vcat, Int[])
ERROR: ArgumentError: reducing over an empty collection is not allowed

julia> reduce(vcat, Int[], init=Int[])
Int64[]
``````
4 Likes

vcat with splatting does not suffer from this problem

``````a=rand(1:10, rand(1:10))
b=rand(1:10, rand(1:10))
c=rand(1:10, rand(1:10))
d=rand(1:10, rand(1:10))
e=rand(1:10, rand(1:10))
f=[]

x=[a,b,c,d,f,e]

vcat(x...)

y=[f,a,b,c,d,e,f]

Base.splat(vcat)(y)
``````

even in terms of performance it does well

``````v=[rand(1:10, rand(0:10)) for _ in 1:100]

using BenchmarkTools
julia> @btime vcat(v...);
873.684 ns (1 allocation: 4.06 KiB)

julia> @btime reduce(vcat, v, init=Int[]);
19.600 μs (102 allocations: 206.09 KiB)

julia> vcat(v...)==reduce(vcat, v, init=Int[])
true
``````

if you notice the number of allocations of `Base.splat(vcat)` is always 1 up to vectors with 256 elements. While that of `reduce(vcat)` is equal to the size of the vector v + 2

``````
julia> v=[rand(1:10, rand(0:10)) for _ in 1:256]
256-element Vector{Vector{Int64}}:
[8, 4, 8, 1, 1, 9]
[4, 8, 6, 8, 10, 8, 6, 9, 4]
[2, 3, 10, 5, 2]
[6]
[2, 3]
[5, 4]
...
[8, 4, 4, 10, 9, 1]
[5]
[8, 9]
[3, 10, 9, 3, 3, 6, 9, 1, 9]
[]

julia> @btime vcat(v...);
2.044 μs (1 allocation: 9.94 KiB)

julia> @btime reduce(vcat, v, init=Int[]);
120.400 μs (258 allocations: 1.29 MiB)
``````
1 Like

For some reason, the `init` slows it down quite a bit:

``````julia> @btime reduce(vcat, \$v, init=Int[]);
23.151 μs (101 allocations: 210.23 KiB)

julia> @btime reduce(vcat, \$v);
962.444 ns (2 allocations: 5.12 KiB)
``````

There is a related open issue for `mapreduce` (#31137) — it seems like we didn’t optimize a sufficiently low-level method of `reduce`.

4 Likes