How to "unshare" an array?

General Usage
1

I want to efficiently build up a large matrix in a loop. At every iteration I will:

  1. Perform some linear algebra operations on the matrix (it’s more complicated than the example below), and
  2. Append a new row to the matrix.

The problem I have is that whenever I reshape my array buffer into a matrix, it gets marked as “shared”, even though the matrix representation is temporary and goes out of scope.

Is there a way to “unshare” the array? Or is there a better approach to this problem?

julia> x = rand(2);

julia> for _ in 1:10000
        println(reshape(x, 2, :) |> y -> y * y')
        append!(x, rand(2))
       end
[0.009747815392725486 0.039514699187714185; 0.039514699187714185 0.16018065474042284]
ERROR: cannot resize array with shared data
Stacktrace:
 [1] _growend!
   @ ./array.jl:922 [inlined]
 [2] append!(a::Vector{Float64}, items::Vector{Float64})
   @ Base ./array.jl:1019
 [3] top-level scope
   @ REPL[658]:3
2

looks like a bug, please file an issue on github

3

I think the relevant issue is here.

2 Likes
4

this seems to be a weaker version, i.e. the previous reshape is never gonna be used again, we should be able to prove that.

5

Efficiently build up a matrix? Allocate the whole matrix in one operation…

6

Good point. But I was also thinking of use cases where I want to limit the matrix to say 10,000 rows. So after reaching 10,000 rows, for each iteration before every push, I’d popfirst to maintain the size.

7

I was recommended ElasticArrays.jl when I had a similar question and it worked wonders for me.

1 Like
8

Or use a circular buffer (matrix). Aka queue. Julia has one (https://github.com/Vexatos/CircularArrays.jl).