# Efficient reshaping of 2D array by pairs of columns

Hi,

I’m building up an MC code that has to run fast and efficiently. At one step I need to reshape a large array with two columns into an array with much less columns and many more rows. A very simple example woudl be that I want this

``````9×2 Array{Int64,2}:
1  2
3  4
5  6
7  8
9 10
11 12
13 14
15 16
17 18
``````

to become like this

``````3×6 Array{Int64,2}:
1  2   7   8  13  14
3  4   9  10  15  16
5  6  11  12  17  18
``````

Is there any simple and efficient way to do this kind of reshaping? I mean, with no intermediate array creation, and such that it is fast?

Thanks a lot,

Ferran.

Check out the `reshape` function.

Thanks, yes. I was actually asking about how to reshape not along one axis (=one column), but by pairs of columns. Notice in my actual problem I have an array of (NpNw) x 2 elements that I want to reshape into Np x (2Nw) form…

It’s unlikely you’ll find a way to do that without modifying the data in memory, since the Array type stores 2d arrays in column-major order. The first array will have 1 3 5 7 9 11 … in contiguous memory, the second will have 1 3 5 2 4 6 … in contiguous memory. To go from on to the other you’ll have to modify the contents of memory. You might be able to do this with a sequence of reshapes and transposes.

1 Like

You could probably eliminate the inner `permutedims` and maybe even the initial `reshape` by accumulating into a `(2, 3, N)` array to start with, but to get the final result you do have to move values around in memory.

``````julia> aa = [1  2
3  4
5  6
7  8
9 10
11 12
13 14
15 16
17 18]

julia> reshape(permutedims(reshape(permutedims(aa), (2, 3, 3)), (2,1,3)), (3, 6))
3×6 Array{Int64,2}:
1  2   7   8  13  14
3  4   9  10  15  16
5  6  11  12  17  18
``````
1 Like

Here’s one way to write the reshapes & permutations:

``````julia> using TensorCast

julia> @cast B[a,(c,b)] := A[(a,b),c]  a:3
3×6 reshape(PermutedDimsArray(::Array{Int64, 3}, (1, 3, 2)), 3, 6) with eltype Int64:
1  2   7   8  13  14
3  4   9  10  15  16
5  6  11  12  17  18
``````

Edit – This expands out to the following, with one permutation:

``````sz_a, sz_c = 3, size(A, 2)
tmp = PermutedDimsArray(reshape(A, (sz_a, :, sz_c)), (1, 3, 2))
B = reshape(tmp, (sz_a, :))
``````
4 Likes

@contradict, how would you write your solution in terms of the `Np` and `Nw` variables defined by Ferran Mazzanti? Thank you.

Oh, thank you guys for the solutions. They seem to be fairly involved, so I’ll have to study them carefully…
Best,
Ferran.

Maybe like this?

``````rearrange(aa, Np, Nw) = reshape(permutedims(reshape(permutedims(aa), (2, Np, Nw)), (2,1,3)), (Np, 2*Nw))
``````

First, get the pair alone. Then swap the two dimensions that need to be moved in memory. Then collapse out the added dimension.

In real code, it would probably be worth ensuring the function produces a sensible error when `aa` has an unexpected shape.

Although, now that I know about TensorCast, I’d probably use that!

1 Like

``````reshape(PermutedDimsArray(reshape(A, (Np, :, 2)), (1, 3, 2)), (Np, :))
It does not require `Nw` input …