# OffsetArrays, inbounds, and confusion

**URL:** <https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295>\
**Category:** Internals & Design\
**Tags:** indexing, arrays\
**Created:** [May 19, 2022, 3:50am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295 "2022-05-19T03:50:46Z")\
**Posts on this page:** 14\
**Page:** 3

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [May 20, 2022, 2:07am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/43 "2022-05-20T02:07:55Z")

</div>

oh, so  
`A[index]`  
is “guaranteed” to exist?  
is there a fancy way to do it just with a function `enumerate()` or similar (i.e. to get the counter, index, and value of A in one function?)

` for (counter, index, a) in Xenumerate(A)`

and then if that was the preferred idiom for people to use, offset arrays would “just work” more often then now?

thanks for clarifying

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [May 20, 2022, 3:05am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/44 "2022-05-20T03:05:56Z")

</div>

> pairs(collection)
> 
> Return an iterator over key =\> value pairs for any collection that maps a set of keys to a set of values. This includes arrays, where the keys are the array indices.

Also see this [old answer of mine](https://discourse.julialang.org/t/any-shortcut-to-1-length-myvector/52816/27).

To my knowledge there is no function that combines `pairs` and `enumerate`, but you can just pass the return of `pairs` to `enumerate`.

```julia
julia> using OffsetArrays
[Info: Precompiling OffsetArrays [6fe1bfb0-de20-5000-8ca7-80f57d26f881]

julia> oa = OffsetArray(["a", "b", "c", "d", "e"], -2:2)
5-element OffsetArray(::Array{String,1}, -2:2) with eltype String with indices -2:2:
 "a"
 "b"
 "c"
 "d"
 "e"

julia> for (count, (index, value)) in enumerate(pairs(oa))
           println("$count $index $value")
       end
1 -2 a
2 -1 b
3 0 c
4 1 d
5 2 e

```

---

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [May 20, 2022, 4:49am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/45 "2022-05-20T04:49:18Z")

</div>

thanks. continuing my edification, how come this works to replace traditional `for i=1:length(A)` loop  
`for a in A`  
but not  
`for (i,a) in A`  
or even  
`for (count,i,a) in A`

It seams to me that Julia should have enough info to make this work?

(please be nice, I come from a numerical/Matlab/Fortran background, so all these post 1970s data structures are still slightly opaque).

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [May 20, 2022, 5:09am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/46 "2022-05-20T05:09:33Z")

</div>

You are correct it could act the way you propose, it is a conscious design decision to not go that way. `for a in A` will always iterates just the values of `A`. You could design/change the language to do the equivalent to `for (i, a) in pairs(A)` when you write `for (i, a) in A`, and do the equivalent of `for (count, (i, a))` when you write `for (count, i, a) in enumerate(pairs(A))`. However, this would disallow you to de-structure tuples in the general case, because the syntax would be ambiguous. For example, the following use case would be disallowed (or have its meaning changed):

```julia
julia> A = [('a', 10), ('b', 20), ('c', 30)]
3-element Array{Tuple{Char,Int64},1}:
 ('a', 10)
 ('b', 20)
 ('c', 30)

julia> for (a, b) in A; println("$a $b"); end
a 10
b 20
c 30

```

I do agree with the language design decision here, as I find it to be the more elegant one. Now, any of these iterators that you may want (including the `count, i, a`) can be implemented by enumerators instead.

```julia
julia> enumpairs(xs) = zip(1:length(xs), eachindex(xs), values(xs))
enumpairs (generic function with 1 method)

julia> using OffsetArrays

julia> oa = OffsetArray(["a", "b", "c", "d", "e"], -2:2)
5-element OffsetArray(::Array{String,1}, -2:2) with eltype String with indices -2:2:
 "a"
 "b"
 "c"
 "d"
 "e"

julia> for (count, i, a) in enumpairs(oa)
           println("$count $i $a")
       end
1 -2 a
2 -1 b
3 0 c
4 1 d
5 2 e

```

---

<div class="post-metadata">

**Author:** ![Paulo\_Jabardo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_jabardo/32/3196_2.png) [@Paulo\_Jabardo](https://discourse.julialang.org/u/Paulo_Jabardo)\
**Post date:** [May 20, 2022, 12:54pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/47 "2022-05-20T12:54:54Z")

</div>

**UPDATED. ORIGINAL SOLUTION WAS WRONG** (won’t delete it for context in the discussion that follows), but I updated this post so that the correct answer is here also (solution proposed by @mbauman).

This was the first thing I thought about when reading this thread. I think I found a decent solution:

```julia
for i in 5:length(a)-4
  #...
end

```

becomes ( **wrong answer** )

```julia
for i in eachindex(a)[5:end-4]
    # ....
end

```

**Correct solution**

```julia
for i in eachindex(a)[begin+4:end-4]
    # ....
end

```

Paulo

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [May 20, 2022, 1:12pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/48 "2022-05-20T13:12:54Z")

</div>

That doesn’t work because the indices of an offset array are _themselves_ offset. So indexing `eachindex(A)[5:end-4]` doesn’t necessarily start from 1 like you’d hope. Instead, you could use `eachindex(A)[begin+4:end-4]`, which has the additional virtue of being symmetrical.

---

<div class="post-metadata">

**Author:** ![ederag](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ederag/32/4106_2.png) [@ederag](https://discourse.julialang.org/u/ederag)\
**Post date:** [May 20, 2022, 4:46pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/49 "2022-05-20T16:46:06Z")

</div>

~~For generic code I’d refrain from using `[begin+4:end-4]`,  
as it would fail on “strided” arrays (e.g. arrays stored in compact form, but with only even indices).  
~~

_Edit: leaving the part below, which shows how indexing on `eachindex` surprised me._

I read Paulo’s post as “iterate from the 5th element to the 4th before last”,  
which seems a valid use case.

To do that, one can’t index on the `eachindex(oa)` iterator itself

```julia
oa = OffsetArray(["a", "b", "c", "d", "e"], -2:2)
eachindex(oa)[3:5]
BoundsError: attempt to access 5-element OffsetArrays.IdOffsetRange{Int64, Base.OneTo{Int64}} with indices -2:2 at index [3:5]

```

I currently do not understand what it is trying to achieve.

But on the collected indexes, it works nicely:

```julia
oa = OffsetArray(["a", "b", "c", "d", "e"], -2:2)

# loop on the 3rd to 5th element
for idx in collect(eachindex(oa))[3:5]
	println("$idx: $(oa[idx])")
end

0: c
1: d
2: e

```

I’d expect `Base.iterators.take(iterator, 3:5)` to return a nice iterator for this purpose,  
but it’s not the case yet (as of julia-1.6.5).

Julia is already awesome, and progressing quickly in the right direction. Keep on the good work !

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [May 20, 2022, 4:49pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/50 "2022-05-20T16:49:41Z")

</div>

> [@ederag](#):
>
> as it would fail on “strided” arrays (e.g. arrays stored in compact form, but with only even indices).

This is false. All `AbstractArray` subtypes must have `AbstractUnitRange` axes — even those with nontraditional indexing. [Interfaces · The Julia Language](https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-array)

Your **memory** can be strided, but the **indices** in Julia must be contiguous.

---

<div class="post-metadata">

**Author:** ![ederag](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ederag/32/4106_2.png) [@ederag](https://discourse.julialang.org/u/ederag)\
**Post date:** [May 20, 2022, 5:05pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/51 "2022-05-20T17:05:49Z")

</div>

> All `AbstractArray` subtypes must have `AbstractUnitRange` axes

Thanks. I see now that my hypothetic “strided” array could not be an `AbstractArray`.  
This simplifies things.

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [May 20, 2022, 5:54pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/52 "2022-05-20T17:54:59Z")

</div>

Other than `OffsetArrays`, there are many containers that support only a part of array interface, like Queue, Stacks and so on.

So, If I have a function that only pushes an element to its argument, I just wat to be sure to dispatch any argument type that is `push`-able. But there is no such trait-based dispatch.

On the other side, looks like AbstractArray is a poor abstraction in terms of its “compactness” / “surface area” (brings too many assumptions and methods)? In other words, a single abstract type bounds too many traits together.

So, maybe an ideal solution (for Julia 2.0, who knows?) would be to have any abstract type declared as a set of small traits? Then, type dispatch over types can be expanded to dispatch over custom sets of traits, so something like this would be a valid code for any subtype, that supports all declared traits:

```julia
# trait set for A is a subset of AbstractArray, but not OffsetArray:
function my_sum(A::{Indexable, IndexFromOne, HasLength, FiniteLength, Eltype{T}, etc.}) where T
    r = zero(eltype(A))
    for i in 1:length(A)
        @inbounds r +=A[i]
    end
    return r
end

```

---

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [May 20, 2022, 7:46pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/53 "2022-05-20T19:46:33Z")

</div>

Thanks! I didn’t know about `zip()`.

Putting my numerics hat on and to clarify… If i was to use `enumpairs()` in “matlab” style Julia (i.e vectors and matrix of Float64’), Julia is smart enough to make this as fast as if I wrote it in the “Fortran” style

```julia
count = 1
for i=1:N
  a = A[i]
  ...
  count = count + 1
end

```

i.e. there is no numerical performance hit by using “fancy” functions like `zip()` and `enumerate()` rather than “raw” `for` loops.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [May 20, 2022, 8:06pm UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/54 "2022-05-20T20:06:53Z")

</div>

If the version with `enumerate`/`pairs` are slower, it’s a bug.

---

<div class="post-metadata">

**Author:** ![Paulo\_Jabardo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_jabardo/32/3196_2.png) [@Paulo\_Jabardo](https://discourse.julialang.org/u/Paulo_Jabardo)\
**Post date:** [May 21, 2022, 12:01am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/55 "2022-05-21T00:01:45Z")

</div>

I tried it out quickly and it appeared to work… Must have typed something wrong. Sorry

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [July 10, 2022, 7:27am UTC](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295/56 "2022-07-10T07:27:50Z")

</div>

> [@simsurace](#):
>
> Isn’t it a concern that `require_one_based_indexing` isn’t exported? My understanding was that unexported functions are not part of the public API.

Whether a function is exported or not does not by itself indicate whether it is public API or not. The main indicator if it is public API is whether it had a docstring and is included in the manual. If it is documented and the documentation does not say otherwise, it is public API.

There have been some efforts to explicitly declare components to be public API. One example is this issue:

> <https://github.com/JuliaLang/julia/issues/42117>
>
> \## Summary
> 
> I would like to give package developers a way to declare names tha…t are part of their package's public API without needing to \`export\` them.
> 
> This feature request has two parts:
> 1. The \`Base.@public\` macro, which declares a name as public without \`export\`ing it.
> 2. The \`Base.public\_not\_exported(m::Module)::Vector{Symbol}\` function, which returns the list of all non-exported public names in the module \`m\`.
> 
> This feature request is non-breaking. Therefore, it can be implemented in a Julia 1.x release.
> 
> \## Part 1: \`Base.@public\` macro
> 
> For example, suppose that my package has a public function named fit. Because that word is so common, I don't want to export it from my package. But I want to indicate that it is part of the public API in a structured way that downstream tools (Documenter.jl, JuliaHub, editors/IDEs, etc.) can understand.
> 
> So in this example, I could do something like the following:
> 
> \`\`\`julia
> module Foo
> 
> export cool\_stuff # public and exported 
> @public fit # public but not exported 
> 
> function cool\_stuff end
> function fit end
> 
> function private\_stuff end # private 
> 
> end # module
> \`\`\`
> 
> \## Part 2:\`Base.public\_not\_exported\` function
> 
> The signature of the function will be: \`Base.public\_not\_exported(m::Module)::Vector{Symbol}\`.
> 
> Example usage:
> 
> \`\`\`julia
> julia\> Base.public\_not\_exported(Foo)
> 1-element Vector{Symbol}:
> :fit
> \`\`\`
> 
> \## Related Discussions
> 
> There is some related discussion in https://github.com/JuliaDocs/Documenter.jl/pull/1507. However, I really want this to be something that any tool can use, so I don't want it to be specific to Documenter.

And this package:  
[https://docs.juliahub.com/PublicAPI/Jfs06/0.1.0/](https://docs.juliahub.com/PublicAPI/Jfs06/0.1.0/)

[Previous page](https://discourse.julialang.org/t/offsetarrays-inbounds-and-confusion/81295.md?page=2)