Initialize array to concatenate in a for loop

New to Julia
,
1

Hi there,

I am trying to concatenate arrays in a for loop. However, I am unsure of how to initialize the array as I keep getting dimension errors. Please see my code below and the error I get.

allArrays = Float64[]  ## 0-element Array{Float64,1}
for i in 1:j
    start_index = trunc(Int, idx_en11[:,i][1])
    stop_index = trunc(Int, idx_en12[:,i][1]).-1 
    myArray = avg[start_index:stop_index]                      
    allArrays = hcat(allArrays, myArray)
end

Error: DimensionMismatch(“vectors must have same lengths”)

The sizes of allArray is (0,) and myArray is (447,). How can I initialize allArray so I can hcat it with myArray?

Thank you!

2

I think you probably want to use vcat.

julia> hcat(rand(5), rand(4))
ERROR: DimensionMismatch("vectors must have same lengths")
Stacktrace:
 [1] hcat(::Vector{Float64}, ::Vector{Float64})
   @ Base ./array.jl:1804
 [2] top-level scope
   @ REPL[2]:1

julia> vcat(rand(5), rand(4))
9-element Vector{Float64}:
 0.8453715926507095
 0.9603306918076744
 0.8382656481984647
 0.11846680775129415
 0.3275371598404506
 0.8907363377369538
 0.46712429010796463
 0.3781140405637613
 0.7716240800560084
1 Like
3

Alternatively, perhaps you are trying to do this:

julia> allArrays = Matrix{Float64}(undef, 447,0)
447×0 Matrix{Float64}

julia> myArray = rand(447);

julia> allArrays = hcat(allArrays, myArray); size(allArrays)
(447, 1)

julia> allArrays = hcat(allArrays, myArray); size(allArrays)
(447, 2)

julia> allArrays = hcat(allArrays, myArray); size(allArrays)
(447, 3)

julia> allArrays = hcat(allArrays, myArray); size(allArrays)
(447, 4)
1 Like
4

There’s no one neutral element for hcat as your code seems to want. You can make a 447×0 Matrix and do something like hcat(hcat(zeros(447, 0), ones(447)), rand(447)).

But applying hcat iteratively like this isn’t ideal, it needs O(N^2) memory, better to do it once. Some ways:

allArrays = reduce(hcat, [avg[trunc(Int, idx_en11[1,i]), trunc(Int, idx_en11[1,i])-1] for i in 1:j])

myArrays = map(1:j) do i
    start_index = trunc(Int, idx_en11[1,i])
    stop_index = trunc(Int, idx_en12[1,i]) - 1 
    @view avg[start_index:stop_index]
end
allArrays = reduce(hcat, myArrays)

using Compat
stack(1:j) do i
    start_index = trunc(Int, idx_en11[1,i])
    stop_index = trunc(Int, idx_en12[1,i]) - 1 
    @view avg[start_index:stop_index]
end
4 Likes
5

FYI, this is not a good way to access an element of idx_en11, since it first creates a full temporary vector and then reads out its first element.

Make sure instead to write

idx_en11[1, i]

to directly access the correct element, like in @mcabbott’s post.

1 Like
7

Ah thank you for letting me know!

8

Thank you, for your feedback.

I realize I took some important parts out of my original code when I was trying to simplify it to post. myArrays are of different lengths. Before I was trying to concatenate them into a 2D array, I was checking the length of the 1D arrays and filling the array with Nan values so all arrays would be the same length.

        if size(myArray)[1] < 447
            n = 447 - size(myArray)[1]
            nanArray = fill(NaN, (n, ))
            myArray = vcat(myArray, nanArray)
        end

In your provided solutions, it doesn’t seem to work with different length arrays and I’m not sure how I would incorporate adding Nans to make the array lengths the same.

Is there a way to stack 1D arrays of different lengths to make a 2D array that will automatically fill the missing values with Nan values?

9

The built-in cat functions don’t handle this for you. One way is just to write a loop, this should be fast:

out = fill(NaN, 447, cols)  # makes a Matrix{Float64}
for col in 1:cols
    a = trunc(Int, idx_en11[1,col])
    b = trunc(Int, idx_en12[1,col]) - 1
    len = b - a + 1
    out[1:len, col] .= @view avg[a:b]
end

I’m sure there are many package solutions if you want a less imperative more functional look. In fact I wrote one at some point, this would be raggedstack(myArrays, fill=NaN) from here, after myArrays = map(... above.

1 Like
10

Thank you, I will give this try!

11

If would contemplate just push each new vector into a vector of vectors.

allArrays = Vector{Float64}[]  ## 0-element Vector{Vector{Float64}}
for i in 1:j
    start_index = trunc(Int, idx_en11[:,i][1])
    stop_index = trunc(Int, idx_en12[:,i][1]).-1 
    myArray = avg[start_index:stop_index]            
    push!(allArrays, myArray)          
end

Here a demo of those mechanics:

julia> allArrays = Vector{Float64}[]
Vector{Float64}[]

julia> push!(allArrays, rand(447))
1-element Vector{Vector{Float64}}:
 [0.5553100384709726, 0.14329276653838385, 0.2122371540922311, 0.19173683594235424, 0.6721354076745469, 0.6338905566453745, 0.21436969177264487, 0.6177553797319678, 0.19714812767752599, 0.6355347292448187  …  0.8901893528103014, 0.3466308206560611, 0.6672522121273377, 0.5466765076644736, 0.507350924539655, 0.972234282646458, 0.6049229046377905, 0.8010342902465192, 0.9842372301506985, 0.9268759753477283]

julia> push!(allArrays, rand(558))
2-element Vector{Vector{Float64}}:
 [0.5553100384709726, 0.14329276653838385, 0.2122371540922311, 0.19173683594235424, 0.6721354076745469, 0.6338905566453745, 0.21436969177264487, 0.6177553797319678, 0.19714812767752599, 0.6355347292448187  …  0.8901893528103014, 0.3466308206560611, 0.6672522121273377, 0.5466765076644736, 0.507350924539655, 0.972234282646458, 0.6049229046377905, 0.8010342902465192, 0.9842372301506985, 0.9268759753477283]
 [0.3948109027175797, 0.17318306720945775, 0.10221023595881262, 0.9718289233265024, 0.10302272546278368, 0.07161687819824236, 0.5311699483090109, 0.7489144436217869, 0.3962305676607649, 0.774907278037163  …  0.8169160812295143, 0.5272617486552499, 0.13392091736481082, 0.5307152194931672, 0.7172582090369259, 0.9004315541474207, 0.12043996279926983, 0.9667534635773031, 0.2982939926425868, 0.6200865956828158]

julia> push!(allArrays, rand(10))
3-element Vector{Vector{Float64}}:
 [0.5553100384709726, 0.14329276653838385, 0.2122371540922311, 0.19173683594235424, 0.6721354076745469, 0.6338905566453745, 0.21436969177264487, 0.6177553797319678, 0.19714812767752599, 0.6355347292448187  …  0.8901893528103014, 0.3466308206560611, 0.6672522121273377, 0.5466765076644736, 0.507350924539655, 0.972234282646458, 0.6049229046377905, 0.8010342902465192, 0.9842372301506985, 0.9268759753477283]
 [0.3948109027175797, 0.17318306720945775, 0.10221023595881262, 0.9718289233265024, 0.10302272546278368, 0.07161687819824236, 0.5311699483090109, 0.7489144436217869, 0.3962305676607649, 0.774907278037163  …  0.8169160812295143, 0.5272617486552499, 0.13392091736481082, 0.5307152194931672, 0.7172582090369259, 0.9004315541474207, 0.12043996279926983, 0.9667534635773031, 0.2982939926425868, 0.6200865956828158]
 [0.9697403100437177, 0.9100866635975313, 0.1480070250228125, 0.004596600503084303, 0.5184083549582552, 0.6072774948973072, 0.46150602234938054, 0.5946353696886346, 0.9346405608456394, 0.463381683256588]

julia> allArrays
3-element Vector{Vector{Float64}}:
 [0.5553100384709726, 0.14329276653838385, 0.2122371540922311, 0.19173683594235424, 0.6721354076745469, 0.6338905566453745, 0.21436969177264487, 0.6177553797319678, 0.19714812767752599, 0.6355347292448187  …  0.8901893528103014, 0.3466308206560611, 0.6672522121273377, 0.5466765076644736, 0.507350924539655, 0.972234282646458, 0.6049229046377905, 0.8010342902465192, 0.9842372301506985, 0.9268759753477283]
 [0.3948109027175797, 0.17318306720945775, 0.10221023595881262, 0.9718289233265024, 0.10302272546278368, 0.07161687819824236, 0.5311699483090109, 0.7489144436217869, 0.3962305676607649, 0.774907278037163  …  0.8169160812295143, 0.5272617486552499, 0.13392091736481082, 0.5307152194931672, 0.7172582090369259, 0.9004315541474207, 0.12043996279926983, 0.9667534635773031, 0.2982939926425868, 0.6200865956828158]
 [0.9697403100437177, 0.9100866635975313, 0.1480070250228125, 0.004596600503084303, 0.5184083549582552, 0.6072774948973072, 0.46150602234938054, 0.5946353696886346, 0.9346405608456394, 0.463381683256588]

julia> allArrays[1]
447-element Vector{Float64}:
 0.5553100384709726
 0.14329276653838385
 0.2122371540922311
 0.19173683594235424
 0.6721354076745469
 0.6338905566453745
 0.21436969177264487
 0.6177553797319678
 0.19714812767752599
 0.6355347292448187
 ⋮
 0.3466308206560611
 0.6672522121273377
 0.5466765076644736
 0.507350924539655
 0.972234282646458
 0.6049229046377905
 0.8010342902465192
 0.9842372301506985
 0.9268759753477283

julia> allArrays[2]
558-element Vector{Float64}:
 0.3948109027175797
 0.17318306720945775
 0.10221023595881262
 0.9718289233265024
 0.10302272546278368
 0.07161687819824236
 0.5311699483090109
 0.7489144436217869
 0.3962305676607649
 0.774907278037163
 ⋮
 0.5272617486552499
 0.13392091736481082
 0.5307152194931672
 0.7172582090369259
 0.9004315541474207
 0.12043996279926983
 0.9667534635773031
 0.2982939926425868
 0.6200865956828158

julia> allArrays[3]
10-element Vector{Float64}:
 0.9697403100437177
 0.9100866635975313
 0.1480070250228125
 0.004596600503084303
 0.5184083549582552
 0.6072774948973072
 0.46150602234938054
 0.5946353696886346
 0.9346405608456394
 0.463381683256588
1 Like