# Appending a matrix to a vector of matrices

**URL:** <https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191>\
**Category:** Performance\
**Tags:** performance, intialization\
**Created:** [August 2, 2022, 7:36pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191 "2022-08-02T19:36:09Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gus\_Hart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gus_hart/32/6987_2.png) [@Gus\_Hart](https://discourse.julialang.org/u/Gus_Hart)\
**Post date:** [August 2, 2022, 7:36pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/1 "2022-08-02T19:36:09Z")

</div>

I have a long list (vector) of matrices. They are size 3xn where n is typically 1 \< n \< 7. My long list typically varies between 100K and 1 million. My task is to reduce this list using a rather complicated and expensive equivalence relation (reduction factor is about 100x to 1000x, depending on the case).

My current solution, to identifying the unique matrices, is basically an N^2 search but with as many early loop exits as I can invent. As I accumulate unique matrices, I’m storing them (via `append`) in an _initially_ empty vector. `@time` tells me that 85% of my time is spent in garbage collection. Presumably it’s expensive to append because of frequent reallocations and copying.

Also, I couldn’t figure out how to initialize an empty vector of matrices `{Matrix{Float64}}` so my initial vector is of type `Any`. Maybe that is also trouble?

How would I initialize an empty vector of `{Matrix{Float64}}`? So that I can `append!` or `push!` matrices to it?

Here are some attempts:

```julia
julia> testList = Vector{Matrix{Float64}}
Vector{Matrix{Float64}} (alias for Array{Array{Float64, 2}, 1})

julia> testm
3×3 Matrix{Float64}:
 -0.5 0.0 0.0
  0.0 0.0 0.5
  0.5 0.0 0.5

julia> append!(testList,testm)
ERROR: MethodError: no method matching append!(::Type{Vector{Matrix{Float64}}}, ::Matrix{Float64})
Closest candidates are:
  append!(::DataStructures.MutableLinkedList, ::Any...) at ~/.julia/packages/DataStructures/59MD0/src/mutable_list.jl:160
  append!(::StructArrays.StructVector, ::Any) at ~/.julia/packages/StructArrays/wbEEd/src/tables.jl:27
  append!(::Plots.Series, ::Any...) at ~/.julia/packages/Plots/Ra8fG/src/utils.jl:790
  ...
Stacktrace:
 [1] top-level scope
   @ REPL[276]:1

julia> push!(testList,testm)
ERROR: MethodError: no method matching push!(::Type{Vector{Matrix{Float64}}}, ::Matrix{Float64})
Closest candidates are:
  push!(::Any, ::Any, ::Any) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/abstractarray.jl:2970
  push!(::Any, ::Any, ::Any, ::Any...) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/abstractarray.jl:2971
  push!(::VSCodeServer.JSON.Parser.PushVector, ::Any) at ~/.vscode/extensions/julialang.language-julia-1.6.28/scripts/packages/JSON/src/pushvector.jl:20
  ...
Stacktrace:
 [1] top-level scope
   @ REPL[277]:1

julia> testList = Vector{Matrix{Float64}}[]
Vector{Matrix{Float64}}[]

julia> push!(testList,testm)
ERROR: MethodError: no method matching Vector{Matrix{Float64}}(::Matrix{Float64})
Closest candidates are:
  Array{T, N}(::AbstractArray{S, N}) where {T, N, S} at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/array.jl:563
  Vector{T}() where T at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/boot.jl:476
  Array{T, N}(::StaticArraysCore.SizedArray{S, T, N, M, TData} where {M, TData<:AbstractArray{T, M}}) where {T, S, N} at ~/.julia/packages/StaticArrays/8Dz3j/src/SizedArray.jl:66

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [August 2, 2022, 7:41pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/2 "2022-08-02T19:41:17Z")

</div>

> [@Gus\_Hart](#):
>
> ```julia
> julia> testList = Vector{Matrix{Float64}}
> Vector{Matrix{Float64}} (alias for Array{Array{Float64, 2}, 1})
> 
> ```

You want

```julia

julia> testList = Vector{Matrix{Float64}}(undef, 0)
Matrix{Float64}[]

```

or

```julia
julia> Matrix{Float64}[]
Matrix{Float64}[]

```

This is just a type, not an _instance_ of the type

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 2, 2022, 7:41pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/3 "2022-08-02T19:41:34Z")

</div>

> [@Gus\_Hart](#):
>
> `julia> testList = Vector{Matrix{Float64}}`

This isn’t a Vector, this is a `type`. You must construct an _instance_. Try

```julia
Vector{Matrix{Float64}}()

```

A question, is n constant?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 2, 2022, 7:44pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/4 "2022-08-02T19:44:44Z")

</div>

And you should use `push!`, not `append!`

I actually don’t see how this should allocate a lot, you are just shuffling around references to pre-existing arrays. Most likely, it’s due to the `Vector{Any}` parameter.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 2, 2022, 7:52pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/5 "2022-08-02T19:52:16Z")

</div>

> [@Gus\_Hart](#):
>
> a rather complicated and expensive equivalence relation

Maybe you can speed this up too? If you can use `SMatrix` from StaticArrays.jl. Or improving it otherwise.

---

<div class="post-metadata">

**Author:** ![Gus\_Hart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gus_hart/32/6987_2.png) [@Gus\_Hart](https://discourse.julialang.org/u/Gus_Hart)\
**Post date:** [August 2, 2022, 8:16pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/6 "2022-08-02T20:16:14Z")

</div>

n is constant for each call of the routine. That is, each list has a fixed n

I see now that the error message told me `Vector{Matrix{Float64}}` was a type. For some reason that didn’t click.

I just re-@timed some largish cases and the garbage collection is ~10%. I’ll do some playing around and report back.

@DNF I’ve rarely used `SMatrix`  
types. Maybe now is a good time to learn…

Thanks for the quick responses @DNF and @pdeffebach. I’m stuck at home with covid (it finally got me) so it’s nice to have some help when I’m solo at home.

---

<div class="post-metadata">

**Author:** ![Gus\_Hart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gus_hart/32/6987_2.png) [@Gus\_Hart](https://discourse.julialang.org/u/Gus_Hart)\
**Post date:** [August 2, 2022, 10:16pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/7 "2022-08-02T22:16:39Z")

</div>

> And you should use `push!`, not `append!`

Why is that?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [August 2, 2022, 11:07pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/8 "2022-08-02T23:07:31Z")

</div>

`append!` is meant to append one collection to another. `push!` is meant to add an item to a collection.

---

<div class="post-metadata">

**Author:** ![Gus\_Hart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gus_hart/32/6987_2.png) [@Gus\_Hart](https://discourse.julialang.org/u/Gus_Hart)\
**Post date:** [August 2, 2022, 11:27pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/9 "2022-08-02T23:27:17Z")

</div>

Thanks for the help. I made two more versions. One that didn’t allocate anything extra—it just used a Boolean array to mark the unique matrices. The second one used the proper initialization of the vector of matrices.

Both of these had run-times identical to my initial solution. So, I guess computing the equivalence relation is the bottleneck.

I’m struggling to convert a 3xn array into a static array. Can’t seem to get syntax correct. I can make explicit definition but can’t initialize a static array with a variable:

```julia
@SMatrix [-0.5 0.0 0.0;
         0.0 0.0 0.5;
         0.5 0.0 0.5]
3×3 SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
 -0.5 0.0 0.0
  0.0 0.0 0.5
  0.5 0.0 0.5

julia> testm
3×3 Matrix{Float64}:
 -0.5 0.0 0.0
  0.0 0.0 0.5
  0.5 0.0 0.5

julia> @SMatrix testm
ERROR: Bad input for @SMatrix
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] static_matrix_gen(#unused#::Type{SMatrix}, ex::Any, mod::Module)
    @ StaticArrays ~/.julia/packages/StaticArrays/8Dz3j/src/SMatrix.jl:20
  [3] var"@SMatrix"( __source__ ::LineNumberNode, __module__ ::Module, ex::Any)
    @ StaticArrays ~/.julia/packages/StaticArrays/8Dz3j/src/SMatrix.jl:103
  ...

```

```julia
julia> @SMatrix{3,3}(testm)
ERROR: Bad input for @SMatrix
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33

julia> @SMatrix [-0.5 0.0 0.0;
         0.0 0.0 0.5;
         0.5 0.0 0.5]
3×3 SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
 -0.5 0.0 0.0
  0.0 0.0 0.5
  0.5 0.0 0.5

```

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [August 2, 2022, 11:29pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/10 "2022-08-02T23:29:09Z")

</div>

```julia
julia> SMatrix{3,3}(testm)
3×3 SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
 0.0 0.0 0.0
 0.0 0.0 0.0
 0.0 0.0 0.0

```

---

<div class="post-metadata">

**Author:** ![Gus\_Hart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gus_hart/32/6987_2.png) [@Gus\_Hart](https://discourse.julialang.org/u/Gus_Hart)\
**Post date:** [August 2, 2022, 11:30pm UTC](https://discourse.julialang.org/t/appending-a-matrix-to-a-vector-of-matrices/85191/11 "2022-08-02T23:30:43Z")

</div>

Got it! Thanks! I think the covid is making things harder than normal for me. Thanks again.