There’s a new blog post focused on a relatively new capability in Julia, the ability to use arrays that index starting at values other than 1. The post is aimed at users, giving examples of how this capability can simplify your life in some situations where indexing can get complicated.
In addition to the links in the blog post, it’s probably worth mentioning that there are quite a few helpful resources at JuliaArrays.
This is great!
It helps make much clearer the reasons for wanting to have arbitrary indexing in a language (such as in Lua or Fortran 90, some languages, such as PHP and M/MUMPS arrays are really associative arrays).
One thing the blog doesn’t touch on though is one of the main reasons people have wanted this in the past, which is easier interoperability with most other languages that use 0-based indexes (i.e. C/C++/C#/Objective-C/Swift/Python/Java/JavaScript/Go/Rust/Perl/Haskell/Erlang/Ruby/Delphi Object/Pascal).
Maybe a subsequent blog could touch on that (and on the other wonderful thing you’ve added, PermutedDims) with examples of how to deal with the row-major layout with 0-based indexing arrays of those common languages.
I myself haven’t had the need specifically for 0-based arrays, so I haven’t contributed such a package myself, but I presume others will. (See https://discourse.julialang.org/t/multidimensional-arrays-with-0-based-indices/3325.) As @barche noted, the ZeroRange type in CustomUnitRanges is all ready to go, so it’s not a very hard thing to add an AbstractArray that starts indexing specifically at 0.
To me, it always seemed that the 1 vs 0 debate was vaguely silly, but you’re right that interoperability is an excellent reason to support this. If C returns an int or array of ints that are meant to be the index of another array, it’s nice not to have to always remember to add 1 every time you use it.
I find the 0- vs. 1-indexing to be a matter of index semantics, rather than array semantics. 0-based indices describe lengths or offsets, whereas 1-based indices count elements. Both can describe the same array in a manner that effectively means the same thing. Offset and padded arrays, on the other hand, encode specific semantic information into the array itself. That’s the true power of this new capability. In my view, it’s not an oversight at all that the blog post didn’t touch on 0-indexing.
An alternative is to lexically scope the 0-indexed-ness. That allows you to think in terms of offsets or lengths when appropriate, and still use builtin data structures and algorithms without worrying about how they handle offset arrays. That’s an approach I’ve played with in https://github.com/mbauman/OffsetIndices.jl. It gets even easier to use and extend upon this with the index conversion revamp in 0.6. I had some un-committed code that gets this working with 0.6 that I just pushed. I don’t have a need for this package personally so it’s not been a priority, but I think you may find it interesting, @ScottPJones.
But your OffsetArrays provide the functionality needed to handle 0-based indexing (and a lot more flexiblity), although yes, a custom array type that only did 0-based indexing might be easier to use.
To me, the real silliness was just the insistence that it had to be either one or the other, there are good reasons for both, depending on what you are doing, and as you’ve shown, a lot of good reasons for allowing for arbitrary bases.
I do have a lot of structures that hold the offsets into vectors, used between C/C++ code and Julia, so having this flexibility now is quite useful.
Good point! Thanks for the pointer to your OffsetIndices!
Great blog post. And great packages. Creative indexing and generally the best arrays will be a big thing in Julia.
Going through the post, and running the code, there were a few snags, and here is a list (some snags are perhaps artifacts of my walk-through):
using PaddedViews missing.tfm translates+rotates, but what about translating the head back i.e. adding Translation(125,250) ∘ in front of RotMatrix.Waiting for the next blog post. A blog post with benchmarks on the new indexing schemes will also help clear a way for wider adoption (I guess speed was one of the highest priorities).
The blog post inspired me to share the way I would like to think about offset array indexing:
When implementing a function as a lookup table, one is blurring the distinction between a computational structure and a data structure by mimicking the former with the latter. Offset array indexing seem to be in the same spirit – we seem to be making an interface to the array data structure look more like a computation (which has been cached).
I wonder whether this perspective is familiar to computer scientists and has been expounded anywhere?
As an application, a natural extension of the idea would be to allow an arbitrary “stride” when indexing – the interface would then support arbitrary affine transformations rather than just translations.
Thanks,
this seems to be very useful in several contexts.
Still, could I please ask you to briefly summarise what I can do with an OffsetArray (A)? So far, I guess:
(1) store data, view and manipulate it, like A[0,61] = 1.0
(2) element-wise calculations, like exp.(A)
(3) summing/multiplying along a dimension, like sum(A,1)
(4) but not A’A or A*B, or?
/Paul S