Range(stop)

mkitti · January 17, 2021, 9:24am

Should Julia have a range(stop) similar to the Python range(stop)?

In Julia master (1.7.0-DEV), we recently merged an all keyword range(; start, stop, length, step) in #38041. In addition to allowing all range arguments to be specified by keywords, it enables new combinations of arguments such as range(; stop, length). This also helped to simplify and correct the documentation on range.

Two and three argument positional range(start, stop) and range(start, stop, length) is also on the way in #39228 after approval in #38750.

I have proposed in #39241 allowing for a single-keyword argument range(; stop) and range(; length) as essentially defined as follows:

range(; stop) = 1:stop
range(; stop::Integer) = Base.OneTo(stop)
range(; length::Integer) = Base.OneTo(length)

Python has range(stop), range(start, stop), and range(start, stop, step) where all arguments must be integers.

In #38750, range(stop) was also supported by triage to mean range(start = 1, stop = stop).

I prototyped the following implementation:

range(start; stop=nothing, length::Union{Integer,Nothing}=nothing, step=nothing) =
    _range_positional(start, step, stop, length)

...

range(stop::Integer) = range_stop(stop)

_range_positional(stop::Any    , step::Nothing,      ::Nothing, len::Nothing) =
    _range(nothing, nothing, stop, nothing) # One arg interpreted as `stop`, could be nothing
_range_positional(start::Any    , step::Any    , stop::Any,     len::Any) =
    _range(start, step, stop, len)

...

range_stop(stop) = oneunit(stop):stop
range_stop(stop::Integer) = OneTo(stop)

The main arguments in favor of range(stop):

start = 1 is a reasonable default for one-based index language
range(stop) meaning [1, 2, ..., stop] translates into the rough equivalent of Python’s range(stop) meaning [0, 1, ..., stop-1]. They have a length of stop.
Using Base.OneTo(stop) when an Integer is given providing the same result as eachindex(A) where A is an array.

The main arguments against range(stop):

We already have 1:n which may be more idiomatic.
range(stop) is unexpected after range(start, stop)
If you add keyword arguments, suddenly stop becomes start due to range(start; stop, length, step).

As an alternative to range(stop), I proposed exporting oneto(stop) since the meaning of that seems clearer to me and avoids most of the arguments against range(stop).

The discussion has been bouncing around pull requests for a while, but I thought I would broaden the conversation by bringing it here.

Do you think range(stop) or perhaps oneto(stop) may be useful syntax?

strickek · January 17, 2021, 9:54am

I prefer 1:n for this. Learning Julia (reading code) becomes more difficult if there are to many different solutions for the same thing.

Vasily_Pisarev · January 17, 2021, 11:29am

I’d add to the 3rd argument against that range(stop::Integer) leads to range(5) and range(5.0) suddenly having different behaviors.
oneto(stop) does not have such problems but 1:stop seems more idiomatic.

The one-kwarg versions are fine from my perspective, as they are consistent and do not export new names.

roble · January 17, 2021, 12:04pm

AFAIK, the use of range(stop) is quite common in python. When i came to Julia, it bothered me that the one argument range was not possible. I thought it was unnecessary to define the start argument. Even though one can get used to this, I welcome this change since it makes it easier for new Julia users (that migrate from python) . Of course there are other techniques to index an array or offset array, but range is still regularly used for these purposes.

aplavin · January 17, 2021, 12:04pm

Are these arguments for existence of range(stop), or just for what range(stop) should mean, if it is defined?

Where A is not just an array, but specifically an array of type Array or other one-indexed type.

In python this is often used for array indexing, as all (popular) arrays are zero-indexed. Julia supports arrays with arbitrary indices, thus eachindex(A) is encouraged.

carstenbauer · January 17, 2021, 12:42pm

While it is useful to take the choice of other programming languages as a source of inspiration, it shouldn’t be more than inspiration. After all, we don’t try to form another python. We want to be better than python. IMO, 1:n is just superior to range(n). It is explicit, i.e. doesn’t guess a starting value (although I agree that one(n) is a natural choice), it is specific about types (integer vs float), and, btw, also shorter than range(n).

mkitti · January 17, 2021, 4:07pm

I agree that there is utility to having one solution to accomplishing a task. However, as I study Julia, I end up using tools like eachindex or enumerate more.

1:5 and 1:5.0 has a slightly different behavior.

julia> 1:5
1:5

julia> 1:5.0
1.0:1.0:5.0

julia> 1:5 == 1:5.0 == Base.OneTo(5)
true

julia> eltype(1:5)
Int64

julia> eltype(Base.OneTo(5))
Int64

julia> eltype(1:5.0)
Float64

Making Julia easier to migrate to from Python is a good goal to have. range(stop) does help to migrate programmers and code. It helps one switch automatically from zero to one based indexing. If there is a way to make Julia a little friendlier to the Python user while just adding some redundancy for the Julia user, I think this may be worthwhile.

Both. It is difficult to argue for the existence of range(stop) without declaring what range(stop) is. A strong prerequisite for defining a range with less than three arguments is the presence of reasonable defaults. 1:n for example assumes that step = 1. In a one-based indexing language, start = 1 makes sense as well I would argue and is less arbitrary than something like length = 50.

mkitti · January 17, 2021, 4:23pm

1:n seems borrowed from MATLAB. 1:n does not make Julia better than Python. It just makes it idiomatically distinct.

The longer I use Julia, I end up using forms like 1:n less and less in favor of eachindex or enumerate.

julia> A = Vector{Char}("abcdef");

julia> for i in eachindex(A)
           print("$i ")
       end
1 2 3 4 5 6
julia> for (i,c) in enumerate(A)
           print("$i:$c ")
       end
1:a 2:b 3:c 4:d 5:e 6:f

julia> eachindex(A)
Base.OneTo(6)

I do find it strange that the recommended way, produces a type, Base.OneTo,which you cannot currently create using range or 1:n. That seemed a bit shocking to me when I first found this and made me question if 1:n were truly idiomatic to Julia.

mkitti · January 17, 2021, 4:26pm

To reinforce this point, it’s useful to illustrate what eachindex does.

julia> using OffsetArrays

julia> C = OffsetArray(B, -2:2, -2:2)
5×5 OffsetArray(::Array{Float64,2}, -2:2, -2:2) with eltype Float64 with indices -2:2×-2:2:
 0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0

julia> eachindex(C)
Base.OneTo(25)

julia> D = OffsetArray(A, -6:-1)
6-element OffsetArray(::Array{Char,1}, -6:-1) with eltype Char with indices -6:-1:
 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)
 'b': ASCII/Unicode U+0062 (category Ll: Letter, lowercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)
 'd': ASCII/Unicode U+0064 (category Ll: Letter, lowercase)
 'e': ASCII/Unicode U+0065 (category Ll: Letter, lowercase)
 'f': ASCII/Unicode U+0066 (category Ll: Letter, lowercase)

julia> eachindex(D)
OffsetArrays.IdOffsetRange(-6:-1)

mkitti · January 17, 2021, 4:34pm

Does the existence of eachindex lessen the argument that 1:n is idiomatic or is the one way to do things? eachindex seems to have utility beyond 1:n and is a distinct and perhaps better way to iterate in Julia for many situations.

aplavin · January 17, 2021, 4:47pm

Yes, that’s what I meant - thanks for a concrete example! Indeed, 1:n or proposed range(n) shouldn’t typically be used for array indexing. So their usecases are unrelated to indexing, and it makes much less sense to default to start=1.

mkitti · January 17, 2021, 5:13pm

The way I think of range(stop) is actually range(length) where my expectation is just that I will just get some iterator of the specified length.

If I cared strongly about start or if I wanted to loop over an array, I should probably specify start or do something else.

StefanKarpinski · January 17, 2021, 5:32pm

There are at this point five or six issues and PRs discussion range plus now this thread. This is turning into a denial of service attack on maintainer time and attention. It’s not up for a vote, so I’m not sure what the purpose of starting yet another discussion about it in yet another place is. Let’s please keep the conversation on GitHub on one of the issues where it already is happening. Btw, I am sympathetic to this working but I’m getting very annoyed with all of the different places this is being discussed.

mkitti · January 17, 2021, 5:59pm

This is my fault. I apologize.

However, I don’t think there is an active PR on range(stop) specifically at the moment.

I withdrew my last attempt because I realized this part was still controversial, and I needed to focus on less controversial range(; stop) keyword version.

I came here to understand the remaining controversy. I’ll wait for other PRs on this issue to resolve.

Anyways, please lock this thread if it is a nuisance.

StefanKarpinski · January 18, 2021, 7:41pm

I also must apologize — I was far too brusque and irritable. I know you just want to get the design of positional arguments for range sorted out and I really appreciate your ongoing efforts. I should have held off and not posted when I was cranky. Still, let’s continue the conversation on GitHub.

mbauman · January 19, 2021, 5:54pm

I don’t think the path forward here is individually-discussed method signatures and just trying to find the uncontroversial bits. The path forward is a cohesive design. It’s still not clear to me how you actually want the range function to behave. I know you’re trying to find compromise — and that’s great — but I don’t really want a function designed-by-piecemeal-compromises.

This is why, for example, my review of #39223 started with the docstring. I think you closed and bifurcated that attempt too quickly. Yes, I left a comment that wasn’t in favor, but I’m not Julia. It takes time to develop new features and a consensus — it’s not a race. That’s especially true in cases where we’re really just talking about surface spellings of a function’s methods.

mkitti · January 20, 2021, 1:26am

The funny story behind this is that I am actually pretty happy with how range works in Julia 1.5. My main issue there is the documentation, and my original pull request #37875 in early October was about writing up accurate documentation. Writing that documentation gave me some understanding on how Julia’s range worked, and how it came to be.

By the time I checked in on it in December, I found that it was likely to be obsolete since a number of PRs had emerged to change how range worked and would thus change the documentation. I may have taken @timholy’s encouragement for discussion a bit too far though. Everything I’ve proposed on range stems from ideas I’ve put out there during the discussion.

I agree. My original position was that we do not need a fully positional range syntax. However, if we did have one, I laid out my comprehensive plan for a one, two, and three argument range(length, start, stop) along a “length-oriented” design pattern since length made some sense as a single argument and you could derive non-arbitrary defaults for start and stop from the prior arguments. I mistook the lack of an all positional argument range as an indication that we may be able to design positional range from scratch.

You and others made good points. Some of the points were ones I originally raised in my plan above, so I was highly sympathetic to them myself. Besides questioning start = 1 as a default, I was having some trouble convincing myself that range(stop::Integer) still made sense in the context of range(start, stop, length) and decided to focus on range(; stop) and range(; length) (below). That would be a necessary prerequisite for the other ideas.

github.com/JuliaLang/julia

range(; stop) and range(; length). Single keyword args only. Single pos arg not allowed.

JuliaLang:master ← mkitti:range_one_kw_only

opened 01:45AM - 14 Jan 21 UTC

mkitti

+59 -3

# range(; stop), range(; length) This is a minimalist version of #39223 to ex…pose `Base.OneTo` when a single keyword argument of `range` is given. It only allows either `stop` or `length` to be provided as the sole keyword argument. In that circumstance only, `start`, and `step` are assumed to be one. **It does not allow for one positional argument to be given.** ```julia # Synposis range(; stop) = 1:stop range(; stop::Integer) = Base.OneTo(stop) range(; length::Integer) = Base.OneTo(length) # The above is for explanatory purposes only and not actual code added ``` Usage: ```julia julia> range(; stop = 5.5) 1.0:1.0:5.0 julia> range(; stop = 5) Base.OneTo(5) julia> range(; length = 6) Base.OneTo(6) julia> range(5) # One positional argument is not allowed ERROR: ArgumentError: Cannot construct range from arguments: ... julia> range(; stop=10, length=5) # step is assumed to be 1, no impact from this PR 6:10 julia> range(; step=1, length=5) # start is not assumed to be 1 when more than one keyword is given ERROR: ArgumentError: Cannot construct range from arguments: ... julia> range(; step=1, stop=6 )# start is not assumed to be 1 when more than one keyword is given ERROR: ArgumentError: Cannot construct range from arguments: ... ``` ## Background on Base.OneTo `Base.OneTo` is a highly optimized version of `1:n` that uses the type system to guarantee the first element is one. While `1:n` is very common in Julia, it should be replaced by `Base.OneTo(n)` in highly optimized code. This pull request attempts to make it more accessible via `range`. ```julia julia> using BenchmarkTools julia> o = Base.OneTo(1_000_000_000) Base.OneTo(1000000000) julia> r = 1:1_000_000_000 1:1000000000 julia> @btime $r[$o] 2.499 ns (0 allocations: 0 bytes) 1:1000000000 julia> @btime $r[$r] 4.299 ns (0 allocations: 0 bytes) 1:1000000000 ``` ## References This strictly implements * The preferences of @jw3126 expressed in this comment: https://github.com/JuliaLang/julia/pull/39223#issuecomment-759409885 * It stops just short of implementing the comment by @mbauman and earlier notes from triage because it does not implement a single positional argument, which is objected to by some. It only accepts a single keyword argument. https://github.com/JuliaLang/julia/issues/38750#issuecomment-756391901

To honor Stefan’s wishes, I think we should continue the discussion there if you would like.

NightRider · January 22, 2021, 11:24pm

For what its worth, coming from a non programming background 1:n is way more intuitive. Its the same as R and MATLAB and is more natural in a functional language imo. Also makes sense because it is 1 indexed. And makes it way easier to iterate not starting from 1 which is occasionally needed.