Array indexing

Internals & Design
,
1

Hey Julianners,

Are there a way to ha a syntax like this:

arrofarr = [randn(2,4) for _ in 1:10]
arrofarr[7:end][1:2,3:end] # would select a [4] x [2,2] dim array.

Is this possible? It would be really clean and it could simplify some operation at many place. Also it sounds very intuitive (but of course inbound checks are necessary)

For me I use a function like this:

access_elems(arrs, i) = collect(map(x->x[i], arrs))  # collect is only for timing
access_elems(arrofarr[7:end], [1:2,3:size(arrofarr[1],2)])

which isn’t so nice.

Does anyone know if there is a chance this could be improved later?

2

You could use another comprehension for this:

[ a[1:2, 3:end] for a in arrofarr[7:end] ]
2 Likes
3

You could also broadcast a getindex call, although it’s not as pretty as a comprehension:

 getindex.(arrofarr[7:end], Ref(1:2), Ref(3:4))

VectorOfArray from RecursiveArrayTools may be useful, although it’ll return a 3D array with the vector dimension last rather than a vector of 2D arrays:

julia> using RecursiveArrayTools

julia> arrarr = VectorOfArray([randn(2,4) for _ in 1:10]);

julia> size(arrarr)
(2, 4, 10)

julia> arrarr[1:2, 3:end, 7:end]
2×2×4 Array{Float64, 3}:
[:, :, 1] =
 -1.15581  0.319321
  1.64091  0.576849

[:, :, 2] =
  0.429193  1.08883
 -1.79132   0.3654

[:, :, 3] =
 -0.326053   1.34863
  0.936408  -0.273089

[:, :, 4] =
  1.05441    1.38047
 -0.876187  -0.438934
4 Likes
4

Nice. I actually failed to find the right syntax with getindex and gave up in favour for the comprehension :wink:

5

If you want to use this kind of indexing, presumably the inner arrays are all of the same size, and you probably should represent this data as a multi-dimensional array rather than vector-of-arrays to begin with:

julia> arr = reduce((a,b)->cat(a,b;dims=3), randn(2,4) for _ in 1:10);

julia> arr[1:2,3:end,7:end]
2×2×4 Array{Float64, 3}:
[:, :, 1] =
 0.498871   1.33621
 1.22567   -0.367682

[:, :, 2] =
 -0.663532   0.911144
  0.715724  -0.256871

[:, :, 3] =
 -0.923242  -0.768945
 -0.77289    0.189791

[:, :, 4] =
  0.44265   -0.381229
 -0.996726  -0.991616
2 Likes
6

Thank you for the lot of tip.

For me I can’t really switch to 3D in spite of the facts that I have the same dims. I have batch of 2D matrices in my case and I have to extend it a lot of time.

Thank you for showing the way, how to do the getindex broadcast version. :slight_smile:

So don’t we want to have some support on this array[range1][range2, range3...] It is just beautiful and something that so intuitive, everyone start the syntax with that.

I don’t want to take the emphasis from this topic but are the any plan to support syntactic like:

array[3:] # instead of array[3:end]
array[3:-2] # instead of array[3:end-2]

It is beautiful too and pretty intuitive. :slight_smile:

7

VectorOfArray isn’t actually stored as a contiguous 3D array in memory, it just has a specialized getindex method that lets you treat it like a 3D array. You can still append to it like arrofarr with push!(arrarr.u, rand(2, 4)). ElasticArrays.jl serves a similar purpose in that it allows you to append to the last dimension of an N-dimensional array, but it’s stored contiguously in memory, so you can’t insert/delete elements in the middle like you can with a VectorOfArray.

The array[range1][range2, range3] syntax is unworkable because of Julia’s order of operations: array[range1] is processed first, and that returns a sub-array of the same shape as the original array (in this case, a Vector):

julia> tmp = arrofarr[1:2]
2-element Vector{Matrix{Float64}}:
 [-1.3003492397624834 0.9935985952081343 2.2835928481588357 -2.0331284677618404; 1.0502180694922136 0.5962378647963208 1.332261466006276 0.9387068354558264]
 [-1.1424632759265543 -0.09666017943952224 0.5520569751178473 -0.27689019750702204; 1.602555685315125 2.588662242106899 -0.462705003748125 -0.735344944870217]

You can’t index into a vector as if it’s a 2D array for obvious reasons; you need to broadcast the getindex call for each matrix element of the vector.

julia> tmp[2, 4]
ERROR: BoundsError: attempt to access 2-element Vector{Matrix{Float64}} at index [2, 4]

julia> tmp[1][2, 4]
0.9387068354558264

julia> getindex.(tmp, 2, 4)
2-element Vector{Float64}:
  0.9387068354558264
 -0.735344944870217

There’s a longstanding active proposal to allow index broadcasting with arrofarr[7:end].[1:2, 3:end], and while everybody agrees that it’s nice syntax, there are some corner cases that’ve prevented a decision from being made.

3 Likes
8

That is exactly what I need! How can I send a sign that is it really good idea? :nerd_face: