Using push! to add a new row of data

General Usage
1

What is the proper way of using push to add rows to an existing matrix? I keep getting errors for the below code:

LL = fill(NaN, (0,2));
for j ∈ 1:n #180
    for i ∈ 1:m #360
        push!(LL, [X[i,j], Y[i,j]]);
    end
end

Many thanks!

2

You can only push to vectors, but not to multi-dimensional arrays. (See for example here Resize!(matrix) - #2 by StefanKarpinski why it is this way.)

  • If you really need to add rows dynamically, you could consider using a Vector of Vectors 
LL = Vector{Float64}[]
push!(LL, [1.0, 2.0])

or maybe you could use Introduction · DataFrames.jl or GitHub - JuliaArrays/ElasticArrays.jl: Resizeable multi-dimensional arrays for Julia

  • If you only need to add a few rows/columns, you could use vcat or hcat. For example
LL = [X[:] Y[:]]   # or hcat(X[:],Y[:])  or  hcat(vec(X),vec(Y))
  • If you know in advance how many rows you need, that is of course ideal and it can sometimes give great performance benefits.
LL = fill(NaN, n*m, 2)  # or LL = Array{Float64}(undef, n*m, 2)
k = 1
for j ∈ 1:n #180
    for i ∈ 1:m #360
        LL[k, :] .= (X[i,j], Y[i,j])
        k += 1
    end
end
3 Likes
3

Unfortunately, I got the errors as below:

LoadError: BoundsError: attempt to access 64800-element Vector{Vector{Float64}} at index [1:64800, 2]
Stacktrace:
 [1] throw_boundserror(A::Vector{Vector{Float64}}, I::Tuple{Base.Slice{Base.OneTo{Int64}}, Int64})
   @ Base ./abstractarray.jl:691
 [2] checkbounds
   @ ./abstractarray.jl:656 [inlined]
 [3] _getindex
   @ ./multidimensional.jl:838 [inlined]
 [4] getindex(::Vector{Vector{Float64}}, ::Function, ::Int64)
   @ Base ./abstractarray.jl:1218
 [5] top-level scope
   @ ~/Desktop/Julia_coding/A04_global_projection/XY.jl:41
4

Which method did you try, and can you show your code?

5

If it is ok to push columns instead, you can use GitHub - JuliaArrays/ElasticArrays.jl: Resizeable multi-dimensional arrays for Julia.

6

When you use

LL = Vector{Float64}[]
push!(LL, [1.0, 2.0])

you have to access the rows via LL[1] (first row) or LL[1][2] first row, second element.

1 Like
7

Keep in mind that Julia arrays are ‘column-major’, so adding rows is highly disadvantageous. Can you change your data layout so that you are adding columns instead?

1 Like
8 
LL = Vector{Float64}[]
push!(LL, [1.0, 2.0])
9

That will work. All I need to do is to transpose the matrix after it is generated.

Is there an easier way to do so without relying on an external program package like “Elastic Arrays”? I mean in Matlab, it is as easy as A = [A; B];

10

It’s equally easy in Julia.

m = 360
n = 180
X = rand(m, n)
Y = rand(m, n)
LL = fill(NaN, (0,2));
for j ∈ 1:n #180
    for i ∈ 1:m #360
        LL = [LL; X[i, j] Y[i, j]]
    end
end

The performance of doing it this way is horrible but that’s the case for Matlab as well, or at least it was when I last used Matlab.

1 Like
11

Does anyone know why the code below does not work? Thanks

m = 360
n = 180
X = rand(m, n)
Y = rand(m, n)

LL = fill(NaN, (0,2));
for j ∈ 1:n #180
    for i ∈ 1:m #360
        var = [X[i,j] Y[i,j]];
        global LL = reduce((x, y) -> cat(x, y; dims = 2), [LL, var]);
    end
end
12

Mostly because you try to concatenate along the wrong dimension. But there’s nothing you can gain by using reduce on two elements.

reduce((x, y) -> cat(x, y; dims = 2), [LL, var])

is just a more complicated way, with some amount of overhead, to write

cat(LL, var; dims = 2)
1 Like
13

Or just using concatenation syntax directly:

[LL var]  # cat along dim 2
[LL; var]  # cat along dim 1
2 Likes
14

BTW, using global is discouraged (in all programming languages, not just Julia). Instead, you can put your code inside a function.

Here’s one way:

function mycat(X, Y)
    LL = similar(X, 0, 2)
    for i in eachindex(X, Y)
        LL = [LL; [X[i] Y[i]]]
    end
    return LL
end

But here’s a much better way:

function mycat2(X, Y)
    LL = similar(X, length(X), 2)
    for i in eachindex(X, Y)
        LL[i, 1] = X[i]
        LL[i, 2] = Y[i]
    end
    return LL
end

The last one is 200.000 times faster, and runs in 80us instead of 16 seconds.

2 Likes