for all mat[i,j] I want to check elements to the right/left/top/bottom, but only if that index exists. Is there a more efficient way than using if-statements for start/end columns and rows?
If you’re doing this over all elements in the array, one of the the best ways is by smartly constructing the iteration space. Check out this blog post on writing multidimensional windowed loops:
In many applications, I would add ghost elements at the edges of the array (extra rows and columns of zeros at the edges), and then just loop over the interior. See also Arrays with periodic boundaries
Another advantage of ghost elements is that they are seasonal.