`eachindex(x)` replacement for `1:length(x)-1` and similar

General Usage
21

That’s extremely limiting. It means you can’t even use things like subarrays (views) or ranges.

Trying to express every possible constraint in the type domain exacts a price in flexibility (and complexity of your types). In a dynamically typed culture like Julia’s, often we make the tradeoff of trying to enable more flexibility at the price of requiring testing when new things are combined. Developing a generic, composable library is rarely something you get completely right on the first try — it’s a continual process of improvement as people try things and encounter combinations that don’t work (but should).

(Not all numeric code will work for non-commutative Number types or dimensionful Real values, but that doesn’t necessarily mean you shouldn’t use Number or Real unless you’ve tested with those possibilities. Of course, more rigorous testing is always a plus, but sometimes you want to release something useful even if you haven’t tested every possible combination.)

10 Likes
22

I agree with you: As I mentioned in my post, I would like to use AbstractArray to take full advantage of the generic design of the Julia language and to fit in with the cool kids and all their custom array types.

But I also have a healthy awareness of the fact that I am not an expert coder, and (in cases where it provides a performance advantage) I would like to be able to use things like @inbounds with impunity. With an Array, I know what “contract” I am signing and where the edge cases are; with an AbstractArray, I’m not sure I do (yet). And to reiterate, the concern isn’t merely that someone will pass an MyFunnyAbstractArray and get an out-of-bounds error—the concern is that the code will be silently incorrect and/or leak memory.

Of course, a sensible middle-ground approach is to write your basic internal methods for Arrays only, and then write a public API that converts AbstractArrays as needed. Then the

continual process of improvement as people try things and encounter combinations that don’t work (but should)

that you mention is something that involves modifying only the public API.

4 Likes
23

As I noted in another thread, OffsetArrays provides a view with linear indexes:

OffsetArrays.no_offset_view()

3 Likes
24

I agree with the general thrust of your argument, but my impression is that the reason this and the other thread have gone on for so long is that the terms “generic,” “weird,” and “just work” admit a lot of room for ambiguity. It is evident that many people consider a 0-indexed array implemented via OffsetArrays.jl to be an essential part of their workflow, whereas for others it’s a weird choice given that Julia is a 1-based language.

Yep, it’s all quite subjective. What I was pointing at is rather: if the code is for your personal use, just make any assumptions you want and live a relaxed and happy life. Otherwise, of course, try to be fully generic. This makes for two very different coding experiences, the first casual, easy but non-scalable and meh; the second intellectually much more challenging and beautiful.

But I also have a healthy awareness of the fact that I am not an expert coder, and (in cases where it provides a performance advantage) I would like to be able to use things like @inbounds with impunity. With an Array , I know what “contract” I am signing and where the edge cases are; with an AbstractArray , I’m not sure I do (yet). And to reiterate, the concern isn’t merely that someone will pass an MyFunnyAbstractArray and get an out-of-bounds error—the concern is that the code will be silently incorrect and/or leak memory.

100% agreed, Array means a whole range of things beyond a specific API. You know what implementation it uses, and this information is very often crucial.

This is the challenge we all face with generic coding to higher or lesser degree, regardless of our expertise, I think: limited information. When we write ::AbstractArray we are usually not fully sure what we’re signing up for.

Ok, if you have lots of experience you know the interface by heart, and you know what things you can or cannot do with that variable to have correct code, so your code will likely work quite widely after some tinkering. Invariably, you will need to do tests with a range of unusual inputs to be sure. But one thing I often struggle with when I cannot assume anything about internal implementation is performance. Even if you respect the interface you cannot be sure your implementation will be optimal. For example, if you get a sparse array, you better be careful to avoid setindex!. That can become a huge performance footgun. Even getindex can be much slower than necessary, so you would need to structure your code to take a specialized iteration approach if you happen to have a sparse input. Multiple dispatch is great for this, but this effort takes you into a process of specialization and iteration that often makes your code less and less beautiful and transparent.

So what would Julia 2.0 need to allow us to navigate generic coding with more certainty and aplomb? I’d say we need a way to express the implicit assumptions in our mind about types (regarding API but also performance). So, that means a full-on trait system. But in any case that would still be only part of the solution because you first need to become aware of the assumptions you are making (which is why you seldom see Base.require_one_based_indexing(a) preceding for i in 1:length(a) in the ecosystem).

3 Likes
25

I think that the blame of the underlying issue is more on this sentiment (which I assume is shared by many people) than on array types with unusual indices. There is a reason why Julia does bounds checks by default. I think that one should always feel that using @inbounds is entering danger zone.

4 Likes
26

This sounds really useful. Would it be possible to extend this idea, to provide a one-based integer indexing view into any AbstractArray? Could such support be part of the AbstractArray contract?

1 Like
27

no_offset_view should work with any AbstractArray

Another way to construct a 1-indexed view is by casting the axes of an OffsetArray to UnitRanges. This doesn’t need you to import OffsetArrays :

julia> A = OffsetArray([1,2,3], 1)
3-element OffsetArray(::Vector{Int64}, 2:4) with eltype Int64 with indices 2:4:
 1
 2
 3

julia> B = view(A, UnitRange.(axes(A))...)
3-element view(OffsetArray(::Vector{Int64}, 2:4), 2:4) with eltype Int64:
 1
 2
 3

julia> axes(A)
(OffsetArrays.IdOffsetRange(values=2:4, indices=2:4),)

julia> axes(B)
(Base.OneTo(3),)

This should work with any AbstractArray type as well. The advantage of no_offset_view is that it may unwrap the AbstractArray type if it recognizes that the parent is 1-indexed, whereas view will necessarily add an extra wrapper layer. However, both of these should lead to 1-based indices.

28

Or I think that would be Base.Iterators.take(eachindex(x), length(x)-1)?