Hi,
I have a sparse array with a size of 257 x 195308.
I found that slicing and viewing of a part of the array allocated new memories for every process, and subtraction between two sparse arrays also create another allocations, and the number of iteration made worse the speed and memory allocation, even compared to MATLAB like codes. I feel like viewing a sparse array is not a good choice; it looks like get rid of a benefit of using a sparse array.
So, I want to calculate the sum of absolute difference between two adjacent columns in elementwise ways (for less allocation and high speed) where non-zero value are there.
I came up with an idea that I could use a custom array to calculate non-zero value containing rows.
For example
for i th column, I have [2,4,8] for row information (from sparsearray.rowval as1:257), and [1,4,10,13] for i+1th
I want to create a custom unit-range consisting of a union of [1,2,4,8,10,13] (not an array for this)
using a union function for this purpose takes another times, so the codes were not fast.
Once I can create a custom indices of [1,2,4,8,10,13], I might be able to calculate the sum of the differences at the row of [1,2,4,8,10,13] in a fast way.
Is it possible to create such a custom index?
I’m kind of new to Julia, so JuliaArrays/ CustomUnitRanges.jl is hard to follow.
Thanks!