How to reduce a (reshaped) sparse matrix to a max or sum with acceptable speed?

General Usage Performance
15

On the other hand… I now went back to normal matrices. Earlier, I found that it lasted many seconds as well, but I converted to Float32 which made it allocate much more, and I had not converted to a true 4D matrix, but an array of 3D matrices.

The following is actually way faster than working with sparse arrays:

julia> @time grid4d = reshape(collect(activitygrid),(dims[1],dims[2],dims[3],size(activitygrid)[2]))
  0.605269 seconds (220 allocations: 4.406 GiB, 4.63% gc time)
300×300×36×1460 Array{UInt8, 4}:


function matrixviews(grid4d, ts,xs,ys,zs; view="xy", reduce="max") 
    if reduce == "max"
        if view == "xy"
            return @views dropdims(maximum(grid4d[xs,ys,zs,ts],dims=(3,4)),dims=(3,4))
        end
    elseif reduce == "sum"
        if view == "xy"
            return @views dropdims(sum(grid4d[xs,ys,zs,ts],dims=(3,4)),dims=(3,4))
        end
       # etcetera
    end
end

julia> @btime test = matrixviews(grid4d,:,:,:,:,; view="xy", reduce="max")
  336.462 ms (22 allocations: 88.77 KiB)
300×300 Matrix{UInt8}:

# a selection:

julia> @btime test = matrixviews(grid4d, 100:1100,80:160,100:200,1:36; view="xy", reduce="max")
  65.355 ms (22 allocations: 9.05 KiB)
81×101 Matrix{UInt8}:

I feel like having taken a long unnecessary detour, but happy to be home again. Learned something about sparse matrices along the way.

1 Like