# Efficient reshaping of 2D array by pairs of columns

**URL:** <https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953>\
**Category:** General Usage\
**Created:** [January 6, 2021, 5:48pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953 "2021-01-06T17:48:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ferran\_Mazzanti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ferran_mazzanti/32/7458_2.png) [@Ferran\_Mazzanti](https://discourse.julialang.org/u/Ferran_Mazzanti)\
**Post date:** [January 6, 2021, 5:48pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/1 "2021-01-06T17:48:31Z")

</div>

Hi,

I’m building up an MC code that has to run fast and efficiently. At one step I need to reshape a large array with two columns into an array with much less columns and many more rows. A very simple example woudl be that I want this

```julia
9×2 Array{Int64,2}:
 1 2
 3 4
 5 6
 7 8
 9 10
11 12
13 14
15 16
17 18

```

to become like this

```julia
3×6 Array{Int64,2}:
 1 2 7 8 13 14
 3 4 9 10 15 16
 5 6 11 12 17 18

```

Is there any simple and efficient way to do this kind of reshaping? I mean, with no intermediate array creation, and such that it is _fast_?

Thanks a lot,

Ferran.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [January 6, 2021, 6:06pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/2 "2021-01-06T18:06:24Z")

</div>

Check out the `reshape` function.

---

<div class="post-metadata">

**Author:** ![Ferran\_Mazzanti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ferran_mazzanti/32/7458_2.png) [@Ferran\_Mazzanti](https://discourse.julialang.org/u/Ferran_Mazzanti)\
**Post date:** [January 6, 2021, 6:12pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/3 "2021-01-06T18:12:15Z")

</div>

Thanks, yes. I was actually asking about how to reshape not along one axis (=one column), but by pairs of columns. Notice in my actual problem I have an array of (Np_Nw) x 2 elements that I want to reshape into Np x (2_Nw) form…

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [January 6, 2021, 6:55pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/4 "2021-01-06T18:55:28Z")

</div>

It’s unlikely you’ll find a way to do that without modifying the data in memory, since the Array type stores 2d arrays in column-major order. The first array will have 1 3 5 7 9 11 … in contiguous memory, the second will have 1 3 5 2 4 6 … in contiguous memory. To go from on to the other you’ll have to modify the contents of memory. You might be able to do this with a sequence of reshapes and transposes.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [January 6, 2021, 7:01pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/5 "2021-01-06T19:01:52Z")

</div>

You could probably eliminate the inner `permutedims` and maybe even the initial `reshape` by accumulating into a `(2, 3, N)` array to start with, but to get the final result you do have to move values around in memory.

```julia
julia> aa = [1 2
        3 4
        5 6
        7 8
        9 10
       11 12
       13 14
       15 16
       17 18]

julia> reshape(permutedims(reshape(permutedims(aa), (2, 3, 3)), (2,1,3)), (3, 6))
3×6 Array{Int64,2}:
 1 2 7 8 13 14
 3 4 9 10 15 16
 5 6 11 12 17 18

```

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 6, 2021, 7:02pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/6 "2021-01-06T19:02:06Z")

</div>

Here’s one way to write the reshapes & permutations:

```julia
julia> using TensorCast

julia> @cast B[a,(c,b)] := A[(a,b),c] a:3
3×6 reshape(PermutedDimsArray(::Array{Int64, 3}, (1, 3, 2)), 3, 6) with eltype Int64:
 1 2 7 8 13 14
 3 4 9 10 15 16
 5 6 11 12 17 18

```

Edit – This expands out to the following, with one permutation:

```julia
sz_a, sz_c = 3, size(A, 2)
tmp = PermutedDimsArray(reshape(A, (sz_a, :, sz_c)), (1, 3, 2))
B = reshape(tmp, (sz_a, :))

```

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [January 7, 2021, 10:00am UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/7 "2021-01-07T10:00:54Z")

</div>

@contradict, how would you write your solution in terms of the `Np` and `Nw` variables defined by Ferran Mazzanti? Thank you.

---

<div class="post-metadata">

**Author:** ![Ferran\_Mazzanti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ferran_mazzanti/32/7458_2.png) [@Ferran\_Mazzanti](https://discourse.julialang.org/u/Ferran_Mazzanti)\
**Post date:** [January 7, 2021, 4:10pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/8 "2021-01-07T16:10:13Z")

</div>

Oh, thank you guys for the solutions. They seem to be fairly involved, so I’ll have to study them carefully…  
Best,  
Ferran.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [January 7, 2021, 4:41pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/9 "2021-01-07T16:41:37Z")

</div>

Maybe like this?

```julia
rearrange(aa, Np, Nw) = reshape(permutedims(reshape(permutedims(aa), (2, Np, Nw)), (2,1,3)), (Np, 2*Nw))

```

First, get the pair alone. Then swap the two dimensions that need to be moved in memory. Then collapse out the added dimension.

In real code, it would probably be worth ensuring the function produces a sensible error when `aa` has an unexpected shape.

Although, now that I know about TensorCast, I’d probably use that!

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [January 7, 2021, 4:56pm UTC](https://discourse.julialang.org/t/efficient-reshaping-of-2d-array-by-pairs-of-columns/52953/10 "2021-01-07T16:56:28Z")

</div>

@contradict, thank your for the explanation, learning a lot about this. @mcabbott’s solution is turbo and can be written from his post edit as:

```julia
reshape(PermutedDimsArray(reshape(A, (Np, :, 2)), (1, 3, 2)), (Np, :))

```

It does not require `Nw` input …