Resize!(matrix)

General Usage
1

is it possible to resize an Array with more than one dimension?

For example:

tmp1 = collect(1:6)
@show pointer(tmp1)

for i in 1:5
    resize!(tmp1, length(tmp1) + 2)
    @show pointer(tmp1)

    tmp2 = reshape(tmp1, 2, Int(length(tmp1) / 2))
    @show pointer(tmp2)

    # do something that requires tmp2 as a matrix
    # ...
end

gives me an cannot resize array with shared data error

2

There is not. You have to create a new array and copy data to it, which is what any implementation of this functionality would have to do under the hood. There’s been a few discussions of supporting this, but the fact that it’s inherently so expensive has always steered the decision in favor of making the fact that it’s an expensive operation explicit. Another way to do this is to use a different data structure: a vector of vectors. For that data structure, adding or removing “columns” is cheap.

3

the problem is that a vector of vectors does not have a contiguous memory layout any more.

I solved it exactly the way you described in my special case the size of the matrix does not change very often, so it is not a real problem for me.

5

If I only need to resize the number of columns, and am OK requiring A = unsafe_resize!(A, m), is the following “safe”?

function unsafe_resize!(W::Matrix, m::Integer)
    if m == size(W,2)
        W
    else
        n=size(W,1)
        reshape(resize!(vec(W),n*m),n,m)
    end
end
6

I’m pretty sure it’s not. If the original allocated array is not 1-dimensional, then I think the runtime assumes that it cannot be resized or moved. It would be possible to lift that restriction but it would be considerable work to make sure that’s not being relied on anywhere and that we don’t get performance regressions due to it no longer being guaranteed that a multidimensional array’s bounds won’t change.

7

As long as you just need to resize the last dimension, ElasticArrays.jl might work for you.

1 Like
8

Ah OK, so I guess best to wrap a Vector with a custom matrix type.

1 Like
9

Shameless advertising: I’m preparing a package for this use case: ArraysOfArrays.jl. Most of the coding is done, I hope to have it ready for registration soon (still needs test code and some docs). You’re welcome to give the “dev” branch a try, I’d be happy to get some early feedback. ArraysOfArrays provides two types:

  • ArrayOfSimilarArrays: Provides things like a vector of vectors (all of same length), backed by a single multidimensional array. When backed by an ElasticArray, it supports resize!, etc. For the case of vector-of-vectors, there’s an alias VectorOfSimilarVectors.

  • VectorOfArrays: A vector of arrays (of same number of dinensions, but different size), backed by a single vector. When backed by a standard Base.Vector (the default) it supports push!, etc. For the case vector-of-vectors, there’s an alias VectorOfVectors.

ArraysOfArrays.jl will support UnsafeArrays.@uview.

1 Like
10

Cool! FYI, this is along the same lines as PseudoBlockArray in BlockArrays.jl, though their use cases is slightly different.

11

With PseudoBlockArray, the blocks have the same number of dimensions as the parent, though, right?

1 Like
12

Yes.

13

Thanks for the hint, though, I knew BlockArrays.jl, but I missed PseudoBlockArray - it may come in very handy for some other things I need to do with arrays!

1 Like
14

I’m curious - why does PseudoBlockArray use copies instead of views on getindex?

15

I don’t understand the question: calling getindex should call getindex…

16

Oops, sorry - I meant getblock, not getindex.

17

How does this differ from RecursiveArrayTools.jl? Anything we can/should take up? It’s built specifically for the purpose of growing arrays and tensors one “column” at a time for DiffEq adaptive solutions but VectorOfArrays has been more generally useful.

18

This is a very common operation to perform in high-performance computing. A package providing a matrix type with this functionality would be more than welcome. I asked the same question on SO some days ago.

EDIT: it seems ElasticArrays.jl does the job for resizing the matrix in the last dimension (columns).

19

That’s probably an anachronism: use view(A, Block(1,2)) if you want a view of a block.

20

Thanks!

21

From what I understand (please correct me!), RecursiveArrayTools.VectorOfArray has the underlying data structure Vector{Array}, while ArraysOfArrays.VectorOfArrays is backed by a single, contiguous array. It’s data structure is more similar to SparseArray, internally.

1 Like