I’m looking for a solution to extend a multi-dimensional array along the last dimension (e.g. to store incoming chunks of multi-dimensional data, when the final size is unknown). I didn’t find anything, but here’s my prototype of an ExtendableArray type:
https://gist.github.com/oschulz/6220887859daf1d363bd0073e466b5ea
Does something like this already exist (possibly in more refined form)? If not, are other people interested? I could turn my draft into a package.
Looks pretty good. Because div is slow and size is performance-critical (because it’s used in bounds-checks), you might consider using a Base.SignedMultiplicativeInverse. If you need inspiration, see the Base.ReshapedArray implementation.
Thanks for the feedback, Tim! I’ll improved it a bit, package it up, and hosting at JuliaArrays sounds fine. 
Thanks for the tip! I was worried about that div, but didn’t know about SignedMultiplicativeInverse.
I am highly interested in this. I have my own implementation of lower quality, and it would be great to see this be a package.
I put up a draft here:
https://github.com/oschulz/ElasticArrays.jl/tree/dev
I didn’t want to create it under JuliaArrays directly, in case the name has to change, etc.
@tim.holy (and the other experts), I’d be glad for any comments (here via issues/line-comments). Also, do you have any advice regarding testing? I’d imagine that there’s good testing code for generic arrays around already.
Thank you! I’ve already integrated this into my code, and it is working perfectly.
Sounds great. Regarding testing, I don’t know of a “thoroughly test my AbstractArray type” package (not a bad idea, though). You might look at some of the other JuliaArrays packages for inspiration. Testing things like inferrability of construction (@inferred(ElasticArray{T}(dims))) and indexing (both getindex and setindex!), size checks, bounds checking (@test_throws BoundsError a[0] etc.), iteration (for a in A ... end), perhaps a few general utilities (fill!, copy!, etc.), and anything specific to your package (e.g., append! and prepend!) would be pretty standard.
Regarding the name, I personally think it’s fine as it is.
Nice to hear, @garrison - please let me know if you have any suggestion on how to improve/extend it.
A somewhat heretical question - could we allow the standard Base.Array to be resized along it’s last dimension (e.g. in 0.7)? Or is there some deeper technical issue that prevents it?
Thanks, I’ll get on it, then.
I think there is a deeper issue, though not having looked directly I’m not sure how serious/relevant it is. There are times where the compiler can automatically elide bounds-checks: i.e., you may not have manually added @inbounds, but the compiler can figure out that it would be safe and therefore adds it itself. My understanding is that this analysis becomes harder if we allow growing of anything other than 1-d arrays. But like I said, I’ve never checked that myself.
You might want to click “watch” on your own repository. At least two people have submitted PRs already. Clearly you’re off to a good start 
.
Oops, indeed!
My, you guys were fast, will get on the PR’s tonight.
We do have test/abstractarray.jl, but it’s not really designed for this purpose. It has a whole slew of test_SOMETHING(::Type{<:AbstractArray}) functions that are designed to verify that the abstract fallbacks are working as expected.
I have trouble with the tests, only on OS-X with Julia nightly:
Any hints would be very welcome.
Never mind - turns out I just needed the latest Compat.jl.
All done: GitHub - JuliaArrays/ElasticArrays.jl: Resizeable multi-dimensional arrays for Julia
I tried to add Femtocleaner and Attobot to the repo, but I think I can’t do that myself, under ElasticArrays.
Also, Appveyor testing currently still runs under AppVeyor - who “owns” JuliaArrays on Appveyor, so that we can move testing to AppVeyor?
Concerning registration, any suggestions on initial version number? It seems to work, but is it a 1.0 already?