`eachindex(x)` replacement for `1:length(x)-1` and similar

General Usage
1

I have been following the discussion lately about how functions that take an AbstractArray argument should avoid using the pattern 1:length(x) as an iterator because it is incompatible with interfaces like OffsetArrays.jl.

1:length(x) can be easily replaced with eachindex(x), but what about patterns like 1:length(x)-1, 2:length(x), 1:2:length(x) and so on that are needed occasionally?

Assuming 1D arrays, we have eachindex(A)[begin:end-1]. But this seems kind of clunky, or is it just me? It would be nice (from my very constrained viewpoint) if eachindex had optional arguments that let you “trim” the endpoints, something like eachindex(A, begin, end-1).

  • What’s the best generic replacement for 1:length(x)-1?
8 Likes
2

I think Base.Iterators.take(eachindex(x), 1) might work as a replacement for 1:length(x)-1 compatible with OffsetArrays.jl, but I havent tested it. There are some other handy functions in Base.Iterators you could check out.

1 Like
3

I think it’s important to remember, you don’t need to adapt your own projects to the full AbstractArray specification, only if your package can reasonably be expected to be used with arbitrary array types.

Note that x[(begin+1):(end -1)] doesn’t solve the problem of strided arrays. There is Iterators.take. But x[eachindex(x)[2:(end-1)]] is probably as flexible as it gets.

7 Likes
4

Isn’t this precisely the unsettled issue in the debate, though? Whether supporting arrays with nonstandard indices is part of the “contract” you sign as a developer when you put AbstractArray in your function signature?

5

The way I see it, as long as you don’t use @inbounds, the amount of index errors you get is the measure of whether you need to fix your implementation. If you’re writing an array utility package it matters, but if your package is the final consumer of arrays I don’t really see the issue with letting some things fail.

1 Like
6 
firstindex(x) : lastindex(x) - 1

don’t ask me what to do if indices are not contiguous…

5 Likes
7 
firstindex(x):step(eachindex(x)):lastindex(x)-1
2 Likes
8

idk, what if the valid indicies are:

1, 2, 4, 87
9

AbstractArrays must have AbstractUnitRange axes, so such a situation isn’t worth preparing for.

7 Likes
10

As it was pointed out in slack, this does not cut it for OffsetArrays , as the indices themselves are again an OffsetArray

11

x[eachindex(x)[begin+1:end-1]] is exactly equivalent to x[begin+1:end-1], isn’t it? getindex is going to use eachindex internally anyway.

julia> x = OffsetArray(rand(5), 3)
5-element OffsetArray(::Vector{Float64}, 4:8) with eltype Float64 with indices 4:8:
 0.8182202442166823
 0.8737712267996881
 0.9098827587057597
 0.7470582568095908
 0.05815041488262762

julia> x[begin+1:end-1]
3-element Vector{Float64}:
 0.8737712267996881
 0.9098827587057597
 0.7470582568095908

julia> x[eachindex(x)[begin+1:end-1]]
3-element Vector{Float64}:
 0.8737712267996881
 0.9098827587057597
 0.7470582568095908
4 Likes
12

What’s the best generic replacement for 1:length(x)-1

To me, eachindex(x)[begin:end-1] seems more readable than eachindex(x, begin, end-1). In the first case, it’s obvious that we’re simply looking at a subsection of the indices, whereas in the second case I’ll need to look at the docstring of eachindex to understand what’s happening.

13

Right, but (the bit inside of the x[ ] brackets) eachindex(x)[begin+1:end-1] is not equivalent to begin+1:end-1. I think this is just an imperfectly chosen example on the part of that user. This question concerns the case where you actually need the indices (e.g. to do some kind of Fourier-type transform or moving-average thing that depends on their values), and not just the “slice” of x.

Fair enough.

14

An alternative to eachindex(x)[begin+1:end-1] is UnitRange(eachindex(x))[2:end-1], although I’m unsure if that’s any more readable.

julia> x = OffsetArray(rand(5), 3);

julia> eachindex(x)[begin+1:end-1]
5:7

julia> UnitRange(eachindex(x))[2:end-1]
5:7

julia> eachindex(x)[begin:end-1]
4:7

julia> UnitRange(eachindex(x))[1:end-1]
4:7

The UnitRange axis has the advantage that its indices begin at 1, so it might be easier to deal with.

1 Like
15

The way I see it, as long as you don’t use @inbounds , the amount of index errors you get is the measure of whether you need to fix your implementation.

But if x is a zero-based array, then x[1:length(x)-1] is in bounds! So with or without the @inbounds macro, a calculation based on this index will be silently incorrect rather than throwing an error.

2 Likes
16

I would highly recommend using @view to create a SubArray.

julia> A = collect(1:10)
10-element Vector{Int64}:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10

julia> B = @view(A[begin:end-1])
9-element view(::Vector{Int64}, 1:9) with eltype Int64:
 1
 2
 3
 4
 5
 6
 7
 8
 9

julia> eachindex(B)
Base.OneTo(9)

julia> C = @view(A[begin:2:end])
5-element view(::Vector{Int64}, 1:2:9) with eltype Int64:
 1
 3
 5
 7
 9

julia> C[:] .= -1
5-element view(::Vector{Int64}, 1:2:9) with eltype Int64:
 -1
 -1
 -1
 -1
 -1

julia> A
10-element Vector{Int64}:
 -1
  2
 -1
  4
 -1
  6
 -1
  8
 -1
 10

julia> for i in eachindex(C)
           C[i] = -i
       end

julia> C
5-element view(::Vector{Int64}, 1:2:9) with eltype Int64:
 -1
 -2
 -3
 -4
 -5

julia> A
10-element Vector{Int64}:
 -1
  2
 -2
  4
 -3
  6
 -4
  8
 -5
 10
1 Like
17

Let me emphasise again what was mentioned by @gustaphe above: remember that all the issue with AbstractArray iteration applies to “generic” codebases, i.e. those for which you want the user to be able to plug any weird AbstractArray implementation and have it just work.

The fact that Julia is powerful enough to allow this doesn’t mean at all that every piece of code should take advantage of this power. The delicate issue of full genericness applies more to developers of foundational libraries that aim to be solid and flexible building blocks for more specific applications.
Otherwise, you might want to restrict the type of array you accept, using e.g. Base.require_one_based_indexing, or even just enforce Array as an input, instead of AbstractArray.

[Then again, it’s a great exercise in abstraction (and ambition!) to go full generic if you can. You will learn a lot.]

7 Likes
18

Is there a subtype of AbstractArray that restrict to contiguous indices?

19

Yes, all of them, apparently: `eachindex(x)` replacement for `1:length(x)-1` and similar - #9 by jishnub

2 Likes
20

I agree with the general thrust of your argument, but my impression is that the reason this and the other thread have gone on for so long is that the terms “generic,” “weird,” and “just work” admit a lot of room for ambiguity. It is evident that many people consider a 0-indexed array implemented via OffsetArrays.jl to be an essential part of their workflow, whereas for others it’s a weird choice given that Julia is a 1-based language. Unfortunately, introducing the novel notion of a “generic” codebase does nothing to clarify these issues.

In my (highly uneducated) opinion, generic code is one of the coolest features of Julia, and I think anyone who codes in Julia should aspire to make their package as “generic” as possible to the extent that doing so doesn’t impede performance or introduce safety issues.

To me, the fact that Base.require_one_based_indexing exists implies that any function in a package that doesn’t require 1-based indexing and takes an AbstractArray, must support offset arrays. Note that the docs specifically warn about this:

https://docs.julialang.org/en/v1/devdocs/offset-arrays/#Things-to-watch-out-for

For this exact reason, in my personal code, I typically enforce Array as an input.

4 Likes