I’d like to announce AxisIndices.jl. Briefly, AxisIndices implements a unique interface for indexing and offers a flexible approach to customizing indexing at each axis. The documentation currently provides brief explanations concerning code design and many docstrings with examples. Although this package is young it’s building on some well developed ideas and well used code. I’d also like to note that I’m trying to document and map a well defined path to 1.0. I expect most people interested in the package should be able to get started with the resources I’ve mentioned thus far (if not then please feel free to ask here or if appropriate please create an issue in the github repository). The rest of this post serves as a more informal explanation for why I made this package.
Here’s an example that demonstrates how it can be used for pretty printing.
A while back I found out AxisArrays was likely to be rewritten or superseded by another package in the future (see this). There were several shortcoming of AxisArrays that I kept bumping into so I quickly began participating in discussions concerning the development of alternatives (see this thread for a sample of the many discussions that have taken place). If you read those discussions you’ll find that one of the themes is a modular design where several packages do one thing well and then another package can bind them together to create something even better than AxisArrays. Some notable contributions towards creating solutions have included NamedDims, AxisRanges.jl, IndexedDims.jl, and DimensionalData.jl.
For a long time I contributed pieces of code and tried to involve as many people as I could in further developing solutions for this problem space. I’ve been particularly interested in solutions for the indexing component to AxisArrays. Thus far I’ve made packages that could contribute to a solution for this but wanted to avoid creating my own package because:
There’s already several solutions being developed. Although there is something to be said for exploring multiple solutions, I was concerned that too many different approaches would divide developer attention too much for any single solution to reach sufficient maturity.
There are a lot of very intelligent people in the Julia community that could create this package. My interest in programming is secondary to how it can further my area of research (neuroscience). Therefore, I usually prefer to be a regular contributor than a maintainer for these sorts of projects.
However, I recently realized two things:
The indexing component involves so many different fields (e.g., time analysis, geospatial computing, image analysis, data munging, etc.) that it would be difficult (if not impossible) to reach a completely unified solution for every situation. This means there would either have to be many independent solutions or a single solution that was very generic and flexible. The last option sounded more like Julia and more likely to result in people being invested in a maintainable solution.
Over time I’ve generated a bunch of code that if bound together in a package might start to resemble this solution with fairly little generation of new code.
So I started putting it together at the beginning of February, spending most of my spare time writing documentation and examples for this package. Whenever there was ambiguity over how a method should behave (e.g., vcat, append!) I tried to achieve a sensible default while implementing as many logical points of intervention so that others could alter how a method acts without redesigning everything. I’ve also tried to get as many of the tests into the documentation so that the intended results of each method are readily known by users.
Future Directions
AxisIndices is intended to package a fairly comprehensive set of tools for customizing indexing. After achieving v1.0 I expect changes to be focused mainly on further improving performance and easing customization of indexing. Integration with other packages and implementation of more specialized behavior will likely be pursued in separate packages so that AxisIndices remains as light weight and maintainable as possible. As a point of reference concerning integration with other packages people may be interested in this early effort to bind AxisIndices, NamedDims, and ImageCore.
Ultimately, I want this to be something that benefits the Julia community and look forward to working with others to achieve this.
I would suggest that Ints (and maybe all of <:Integer, or even <:Real) should be not allowed as a key, and should fall back to the default behavior of getindex instead.
This should allow the arrays that use this indexing scheme to be <:AbstractArray, while at the same type supporting key lookups as a convenience feature.
Are you saying you don’t want to have k2 available because it makes something like x[UInt(1)] ambiguous? Currently you can’t use x[UInt(1)] to access keys for that exact reason.
Currently only the following are supported for indexing through the keys like this
AbstractString
AbstractChar
Symbol
Dates.AbstractTime
Everything else (numbers) requires the syntax similar to that used for filtering (see here)
I’m afraid I don’t really understand yet what the purpose of this package is after reading this post and the first pages of the documentation Do you have examples that demonstrate a use case? It seems to me that your post and documentation are geared towards people who are already very familiar with your specific problem space and you focus on implementation details before really explaining the point of it all.
For example, Quick Start starts with “Most users will be able to achieve what they want by just…” but I don’t know what I’d want from the package, yet.
@jules, thanks for the feedback. Is the problem specifically that you don’t understand what I mean by custom indexing or that you don’t understand the purpose of this custom indexing?
@tbeason, when I say filtering keys in the documentation I’m referring to the keys of the axis. So your example would return a all the elements of a where the keys are greater than 2. I’ve found that the terminology here can be a bit tricky so I explicitly define it in the section following quick start. Perhaps that should also be mentioned in the quick start.
I’m looking for a high-level overview what a use case for your package is, where the existing tools fail. I haven’t dealt much with named axes so I don’t know any of the other packages you mentioned. Basically a sales pitch for someone who doesn’t know your problem space, yet. Then I could determine whether it could be useful for me.
Got it. The current examples are more about how it works rather than why it’s useful (I plan on including more practical examples soon). One thought I’ve had that you may be interested in is how this could be used for plotting.
Currently the axes of a plot are a lot like the Axis type in AxisIndices. So maybe you incorporate it in a plot like this.
scene = Scene(axes=(Axis("a", "b", "c"), Axis("A", "B", "C"))
The keys would be the labels you see at each tick and the indices would be where those keys are placed (e.g, “a”, “b”, “c” at positions 1, 2, 3 along the x-axis). In addition to inheriting the methods that manipulate subtypes of AbstractAxis you can now do some nice things with indexing.
For example, maybe the following places a marker at the grid position “b” and “B”.
scene["b", "B"] = circle
This is nice because usually the user doesn’t actually know the numeric coordinates when the axes are much larger.
It has lots of other applications such as referring spatial and time axes.
I’ve updated the “Quick Start” so that it better clarifies terminology and is hopefully better worded. Once I finish my submission for JuliaCon I’ll probably put together an example on how to make a specialized TimeSeries axis. If anyone has specific requests please let me know.
I apologize for continually bumping this but, in response to the feedback I added a quick initial example. You can see some of the results in the OP or find the full example in the “Pretty Printing” section of the docs. Pretty printing isn’t completely finalized yet but it’s starting to look pretty good.
Thank you so much for this — I’ve reinvented something like this at least three times in the past couple years, but never so completely nor with such an elegant interface.
Just yesterday I spent all day fixing some bugs with the way I do this
Thanks! I’ve got a big update I’m working on that should make it a lot easier to make new axis types. Hopefully it’ll be available sometime next week with some nice examples.
AxisIndices is now in the v0.2 series. I’ve also started to expand support and documentation in numerous ways. I think this list covers most of what people will be interested in.
Proliferating abstract types through methods and ensuring they work (makes new custom types very easy to make)
TimeAxis example in docs
Acknowledgement section in docs (If you feel you’re name should be added or want it removed let me know).
MappedArrays.jl support
More Base methods: rot180, rotr90, rotl90, reinterpret, PermutedDimsArray
Finally, I wanted to make clear my intentions with breaking changes in the future. The v0.2.0 change was an early breaking change that I considered necessary to ease the burden of learning how to customize axis behavior. Despite extensive changes to the underlying code base, I don’t really anticipate breaking changes for most users. Most changes were code consolidation and were only breaking because it changed documented (although not exported) code. In the future I’ll make sure any breaking changes will take the form of a PR that I will notify others of before hand, so they can have a say or just stay more informed.
Given that you are proposing this as an interface, breaking changes would only be natural in the design process. Many commonly used interfaces took a lot of time and usage to stabilize.
Fortunately, this particular project has been under discussion in one form or another for about a year (much longer if you count discussion specifically with AxisArrays that evolved into this). Therefore, the actual method of indexing shouldn’t change without a lot more discussion beforehand.
I think the part of the interface that is the most susceptible to breaking changes will be the developer facing stuff (composing new types of axes and array structures). I want to make sure I don’t arbitrarily limit what others are able to do with this package by creating an overly convoluted implementation or adding potentially conflicting features. I’m assuming this part will just require seeing how it does in the wild.
Do you have examples that demonstrate a use case? It seems to me that your post and documentation are geared towards people who are already very familiar with your specific problem space and you focus on implementation details before really explaining the point of it all.