Of course it gives a different row index in general - you are sampling twice? If you want the same index for both arrays, just index them with the same vector?
julia> rows = sample(1:size(A, 1), 2; replace = false, ordered = true)
2-element Vector{Int64}:
2
4
julia> A[rows, :]
2×4 Matrix{Int64}:
2 2 2 2
4 4 4 4
julia> B[rows, :]
2×4 Matrix{Int64}:
6 6 6 6
8 8 8 8