How to efficiently sort subarrays of 2 large arrays in parallel?

General Usage Performance
, ,
1

I have large 1D arrays x and y, and pointers I that separate them into subarrays. My x and y barely fit into RAM and are of different dtypes (one contains UInt32’s, the other Rational{Int64}'s), so I don’t want to join them into a 2D array, to avoid changing dtypes.

For each i in I[2:end], I wish to sort the subarray x[I[i-1],I[i]-1] and apply the same permutation to the corresponding subarray y[I[i-1],I[i]-1]. My attempt:

function sort!(x,y) 
    p=sortperm(x); 
    x[:], y[:] = x[p], y[p]
    end
Threads.@threads for i in I[2:end] sort!() 
    sort!( x[I[i-1], I[i]-1],   y[I[i-1], I[i]-1] ) 
    end

However, already on a small example, I see that sort! does not alter the view of a subarray:

x, y = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(x,y);     println(x,"\n",y)  # works like it should

x, y = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(x[1:5],y[1:5]);     println(x,"\n",y)  # does nothing!!!

Any help on how to do this (as efficiently as possible) are welcome.

For background, I am dealing with data coming from sparse arrays: x=a.rowval, y=a.nzval, I=a.colptr, where a is some SprarseArrayCSC. What I am attempting is writing a custom function subm(a,I,J) that would return the same result as a=a[I,:]; a=a[:,J]; return a, i.e. a submatrix of I-rows and J-columns, but more efficiently. In the special (important) case when I is a permuation, all I need is the above sort! to work as desired, so that entries in each column become sorted. Typically, my matrix is of size 10^6 x 10^6, with between 0 and 10^4 nonzero entries in each column.