Just for completeness, deleteat! is quite efficient and accepts a collection for indices. The indices of the diagonal elements are simply 1:n+1:n^2. Compared to manual loops, it can even be 30% faster.
A = randn(1000,1000);
@btime offdiag($A); # with bounds checking turned off
3.399 ms (2 allocations: 7.62 MiB)
@btime deleteat!(vec($A), 1:size(A,1)+1:size(A,1)^2);
2.595 ms (8 allocations: 7.63 MiB)