I have large 1D arrays x and y, and pointers I that separate them into subarrays. My x and y barely fit into RAM and are of different dtypes (one contains UInt32’s, the other Rational{Int64}'s), so I don’t want to join them into a 2D array, to avoid changing dtypes.
For each i in I[2:end], I wish to sort the subarray x[I[i-1],I[i]-1] and apply the same permutation to the corresponding subarray y[I[i-1],I[i]-1]. My attempt:
function sort!(x,y)
p=sortperm(x);
x[:], y[:] = x[p], y[p]
end
Threads.@threads for i in I[2:end] sort!()
sort!( x[I[i-1], I[i]-1], y[I[i-1], I[i]-1] )
end
However, already on a small example, I see that sort! does not alter the view of a subarray:
x, y = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(x,y); println(x,"\n",y) # works like it should
x, y = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(x[1:5],y[1:5]); println(x,"\n",y) # does nothing!!!
Any help on how to do this (as efficiently as possible) are welcome.
For background, I am dealing with data coming from sparse arrays: x=a.rowval, y=a.nzval, I=a.colptr, where a is some SprarseArrayCSC. What I am attempting is writing a custom function subm(a,I,J) that would return the same result as a=a[I,:]; a=a[:,J]; return a, i.e. a submatrix of I-rows and J-columns, but more efficiently. In the special (important) case when I is a permuation, all I need is the above sort! to work as desired, so that entries in each column become sorted. Typically, my matrix is of size 10^6 x 10^6, with between 0 and 10^4 nonzero entries in each column.