Using end variable in ranges passed to functions other than getindex?


#1

I expected the following to work:

julia> a = [1,2,3,4,5]
5-element Array{Int64,1}:
 1
 2
 3
 4
 5

julia> deleteat!(a, end-1:end)
ERROR: syntax: unexpected "end"

julia> a[end-1:end]
2-element Array{Int64,1}:
 4
 5

I expected it to delete the last two variables in the array, instead it errors out. Any thoughts on why this behavior happens? Is this a bug?


#2

The parser treats end inside square brackets different from end outside square brackets (where it means “end of block” as in for i = 1:10 ... end).


#3

Not quite as succinct, but you could use endof:

deleteat!(a, endof(a)-1:endof(a))

#4

In particular the parser (or rather the “lowering” phase that happens right after parsing) converts any end inside a[...] into a call to endof(a).

You can see this explicitly by calling @code_lowered:

julia> f(a) = a[end-1:end]

f (generic function with 1 method)

julia> @code_lowered f([1,2,3,4])

CodeInfo(:(begin 

        nothing

        return (Main.getindex)(a, (Main.colon)((Base.endof)(a) - 1, (Base.endof)(a)))

    end))

#5

Thanks everyone for the explanations. That makes sense. But is this the best behavior? Could the parser not just recognize ends that are parts of ranges and convert those to Base.endof?


#6

I can’t really answer that question, but I think it’s more complicated.

To expand on Steven’s reply, the lowering of end is context-sensitive. For example, within a multi-dimensional array index, end is lowered to size(a, dim) where dim is determined by the position of end within the index:

Julia-0.6.0> f(a) = a[1, end-1:end, 3]
f (generic function with 1 method)

Julia-0.6.0> @code_lowered f(Array{Int}(3,4,5))
CodeInfo(:(begin
        nothing
        return (Main.getindex)(a, 1, (Main.colon)((Base.size)(a, 2) - 1, (Base.size)(a, 2)), 3)
    end))

#7

I took this too far :smile:: https://github.com/JobJob/EndyIndexes.jl

pkg> add https://github.com/JobJob/EndyIndexes.jl.git
using EndyIndexes

const start_ = StartBasedIdx()
const end_ = EndBasedIdx()

arr = [1:10;]
for idx in (end_-2:end_, start_+1:end_-2)
    @show arr[idx]
end
a = [1,2,3,4,5]
deleteat!(a, end_-1:end_)

Output

arr[idx] = [8, 9, 10]
arr[idx] = [2, 3, 4, 5, 6, 7, 8]
3-element Array{Int64,1}:
 1
 2
 3

Thanks to the fantastic work that went into custom AbstractArray indexing this was surprisingly easy to implement.


Also just re the comment above, in Julia >=0.7, in multi-dimensional index expressions end now lowers to lastindex(a, dim), in single index expressions just to lastindex(a)

jjulia> f1(a) = a[2, end-1:3, 1]
f1 (generic function with 1 method)

julia> f2(a) = a[end]
f2 (generic function with 1 method)

julia> @code_lowered f1(rand(3,3,3))
CodeInfo(
1 1 ─ %1 = (Base.lastindex)(a, 2)                                                                                                                                │
  │   %2 = %1 - 1                                                                                                                                                │
  │   %3 = %2:3                                                                                                                                                  │
  │   %4 = (Base.getindex)(a, 2, %3, 1)                                                                                                                          │
  └──      return %4                                                                                                                                             │
)

julia> @code_lowered f2(rand(3,3,3))
CodeInfo(
1 1 ─ %1 = (Base.lastindex)(a)                                                                                                                                   │
  │   %2 = (Base.getindex)(a, %1)                                                                                                                                │
  └──      return %2                                                                                                                                             │
)

#8

Maybe this is a good point to make advertisement for the idea for 2.0 to use $begin and $end as they are nice and free, unsurprising, and the dollar works as metaphor (the parser is supposed to substitute them by something like endof(a) and $ indicates talking to the parser. )