That is expected, because the access to memory is not contiguous. It can get much worse:
julia> x = rand(100_000);
julia> y = @view x[1:10_000];
julia> @btime sum($y)
944.154 ns (0 allocations: 0 bytes)
5030.276331565986
julia> y = @view x[rand(1:100_000,10_000)];
julia> @btime sum($y)
12.243 μs (0 allocations: 0 bytes)
4984.892231863414
julia>