Pushing to an array which is an element of another array (Modifying an array in an array)

General Usage Performance
1

Imagine the following situation:
I create an array with a fixed size and every element in this array is another array.
In a loop, I want to push tuples to one array of the outer array.
So I want to modify one array which is an element of the outer array.
The following code example is a heavily simplified snippet from my code. Therefor, the code snippet doesn’t do anything useful.

# fill an array full of emty arrays
outer_array = fill([], dim1, dim2, dim3, ...)
# modifiying arrays of outer array (I choosed three for-loops instead of eachindex() because it is more similar to my original code)
for z in 1:dim1, for y in 1:dim2, for x in 1:dim3
       # current solution to modify array
       sub_arr = copy(outer_array[z, y, x])
       push!(sub_array, (*content in tuple*))
       outer_array[z, y, x] = sub_array
end

This solution is terrible slow, so there must be some better solutions, maybe without copying the array??

2

just push directly to outer_array[z, y, x]?

(if you do sub_array = outer_array[z, y, x], without the copy, that should work to, the assignment won’t copy the sub array).

In parallel, you should not use [], untyped, it is better if you, knowning the type of the tuples, use typeof(tuples)[], to have a concretely typed array.

3

Also, don’t use fill here. It will create an array of references to the same single array. So pushing to one of them, will be visible everywhere.

Use an array comprehension instead.

And, as @lmiq says, [] are not good here. Whenever you see [] in your code, it’s a bad sign.

1 Like
4

Indeed, it is also unnecessary to reassign the subarray. Just replace this

with this single line

push!(outer_array[z, y, x], (*content in tuple*))
5

Thank you (@Imiq , @DNF ) for your quick responses.
I think using fill was the main problem.
My current solution:

outer_array = [[] for _ in 1:dim1, _ in 1:dim2, _ in 1:dim3]
for z in 1:dim1, for y in 1:dim2, for x in 1:dim3
       push!(outer_array[z, y, x], (*content in tuple*))
end

That works, but I don’t have type annotations yet. So how can I get rid of [] in the array comprehension and how can I assign the type Tuple to []?

6

I’m not sure if this has any significant impact here, but as Julia is column-major, the inner/fastest changing loop variable should correspond to z, and it seems to be x in your example. Try swapping the variables in the for loops.

7

Thank you, too. I already heard about Julia’s column-major order, but I never really paid attention to it.
So do you mean something like should be faster?

for x in 1:dim3, for y in 1:dim2, for x in 1:dim1
       push!(outer_array[z, y, x], (*content in tuple*))
end

Do I have to change something else to work column-oriented or is not only the order of the loops important?

8

You need to correct the typo (it should be z in the inner loop) and AFAIK, yes, that is the recommended loop order for Julia multidimensional arrays.

9

What is the type of (*content in tuple*)? Prepend that to []. For example, if it’s Tuple{Float64, Int}, you write

outer_array = [Tuple{Float64, Int}[] for _ in 1:dim1, _ in 1:dim2, _ in 1:dim3]

Elements that are consecutive in the first dimension will be consecutive in memory. But in this case, the elements are references to other arrays, located elsewhere (and those element will be consecutive), so in this case it is unlikely to make difference. Can’t hurt, though.

2 Likes
10

Thank you all for your answers. Now, the real code in my project works aswell.
The main problem was that I used fill (post #3).

11

An easier way to write this loop is

for zyx in eachindex(IndexCartesian(),outer_array)
       z,y,x = Tuple(zyx) # only need this if you want to extract the individual indices
       push!(outer_array[zyx], (*content in tuple*)) # could also index with [z,y,x]
end

This will work even if you change the number of dimensions in your array (except the z,y,x = Tuple(i) part won’t) or if you use an array with unusual indexing. This will also use an index order that is good for most standard Julia arrays.

EDIT: thanks to the commenter below for pointing out that I had originally written CartesianIndex() in my suggestion rather than the correct IndexCartesian().

12

I think you meant eachindex(IndexCartesian(), outer_array) rather than eachindex(CartesianIndex(), outer_array)

1 Like