# Convert a 3D Array to 2D array of vectors

**URL:** <https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755>\
**Category:** General Usage\
**Created:** [August 3, 2021, 5:17pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755 "2021-08-03T17:17:07Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![v-i-s-h](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/v-i-s-h/32/46152_2.png) [@v-i-s-h](https://discourse.julialang.org/u/v-i-s-h)\
**Post date:** [August 3, 2021, 5:17pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/1 "2021-08-03T17:17:07Z")

</div>

Hi,

I have a 3D array (`X::Array{Int8, 3}`, size `N x K x 2`). How can I convert it to a Matrix (size `N x K`) of `Vector` such that new `X` will be `X[i, j] = X[i, j, :]` with very little new memory allocation.

Thanks,  
Vishnu Raj

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [August 3, 2021, 5:30pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/2 "2021-08-03T17:30:16Z")

</div>

Maybe:

```julia
map(CartesianIndices(axes(vol)[1:2])) do i 
    vol[Tuple(i)..., :]
end

```

You can also use views, if that works for your use case (`view(vol, Tuple(i)..., :)`)

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [August 3, 2021, 5:45pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/3 "2021-08-03T17:45:24Z")

</div>

Btw, that sounds like a job for StaticArrays:

```julia
using StaticArrays
vol = rand(1000, 1000, 2)
function test(vol)
    ax = axes(vol)
    map(CartesianIndices((ax[1], ax[2]))) do i
        x, y = Tuple(i)
        SVector(ntuple(z-> vol[x, y, z], 2))
    end
end

```

Which should allocate much less and be much more gc friendly 😉

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [August 3, 2021, 7:12pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/4 "2021-08-03T19:12:03Z")

</div>

@sdanisch, that is pretty advanced code and hard to grasp 🙂

An alternative herein, using [TensorCast.jl](https://mcabbott.github.io/TensorCast.jl/dev/), which seems to use the same “black box technology” under the hood as StaticArrays, but thanks to a very logical and accessible syntax, one can get equivalent results in a more user-friendly way:

```julia
using TensorCast
@cast out2[i,j]{k} := vol[i,j,k]

```

_ **NB:** [curly brackets](https://mcabbott.github.io/TensorCast.jl/dev/options/#Ways-of-slicing) allow different interpretation slices of the same memory, and assignment `:=` returns views_

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [August 3, 2021, 7:17pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/5 "2021-08-03T19:17:13Z")

</div>

Note that if you can cheat a bit (don’t need a `Matrix`, but an `AbstractMatrix` with the same shape will do), you can use essentially no memory at all with StructArrays + StaticArrays

```julia
julia> using StaticArrays, StructArrays, BenchmarkTools

julia> function to_structarray(X::AbstractArray{T, N}) where {T, N}
           return StructArray{SVector{2, T}}(X, dims=N)
       end
to_structarray (generic function with 1 method)

julia> X = rand(Int8, 1000, 1000, 2);

julia> @btime S = to_structarray($X)
  540.741 ns (8 allocations: 368 bytes)
1000×1000 StructArray(view(::Array{Int8, 3}, :, :, 1), view(::Array{Int8, 3}, :, :, 2)) with eltype SVector{2, Int8}:
 [-122, 88] [95, 113] [-66, 65] [37, -3] … [73, -90] [-50, -98] [37, 5] [78, -51]
 [-126, -16] [-54, 5] [94, 38] [78, 84] [-23, -34] [24, -15] [102, 50] [-122, -62]
 ⋮ ⋱
 [-76, -88] [78, 20] [91, -34] [26, 52] [-117, 58] [-60, -95] [-125, -35] [-60, 68]
 [7, 54] [66, 78] [68, -61] [66, -83] [93, -123] [-124, -47] [15, 106] [2, -72]

```

If you want a concrete `Matrix` you can do `collect(S)`, but depending on your application you may not need to.

---

<div class="post-metadata">

**Author:** ![v-i-s-h](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/v-i-s-h/32/46152_2.png) [@v-i-s-h](https://discourse.julialang.org/u/v-i-s-h)\
**Post date:** [August 4, 2021, 7:07am UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/6 "2021-08-04T07:07:44Z")

</div>

Thank you for all your answers. I tried to benchmark the solutions (code below).  
The results were for a `500 x 500 x 2` 3D Array.

```julia
Using StaticArray, by @sdanisch : 4.280 μs (1 allocation: 5.06 KiB)
Using TensorCast, by @rafael.guerra : 515.948 ns (6 allocations: 288 bytes)
Using StaticArrays and StructArrays, by @piever : 403.785 ns (8 allocations: 368 bytes)

```

It seems using TensorCast as suggested by @rafael.guerra gives the best.

Code used:

```julia
using BenchmarkTools

# Using StaticArray, by @sdanisch
using StaticArrays

function method1(X)
    ax = axes(X)

    map(CartesianIndices((ax[1], ax[2]))) do i
        x, y = Tuple(i)
        SVector(ntuple(z -> X[x, y, z], 2))
    end
end

# Using TensorCast, by @rafael.guerra
using TensorCast

function method2(X)
    @cast out[i, j]{k} := X[i, j, k]
end

# Using StaticArrays and StructArrays, by @piever
using StaticArrays, StructArrays

function method3(X::AbstractArray{T, N}) where {T, N}
    return StructArray{SVector{2, T}}(X, dims=N)
end

### Benchmarking
X = rand(Int8, 50, 50, 2);

@btime X1 = method1($X);

@btime X2 = method2($X);

@btime X3 = method3($X);

```

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [August 4, 2021, 8:41am UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/7 "2021-08-04T08:41:18Z")

</div>

Note that if you go for the TensorCast solution (which IIUC under the hood uses `reinterpret` to reinterpret a given block of memory as a matrix of `SVector`s) you should try to have your data in the `2 x N x K` rather than `N x K x 2` format. Then you can just do `reinterpret(reshape, SVector{2, Int8}, X)` to get your matrix, as each `SVector` is stored contiguously in the array as `X[:, j, k]`.

The `N x K x 2` format is a “struct of arrays” format, where the first component of all the arrays `X[:, :, 1]` is stored contiguously, and so is the second component `X[:, :, 2]`.

To see what I mean in practice, experiment benchmarking functions that use the resulting arrays. For example `@btime sum($X2)` (where `X2` was computed with TensorCast) should be substantially slower than `@btime sum($X1)` or `@btime sum($X3)` (which hopefully should be very similar).

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [August 4, 2021, 1:34pm UTC](https://discourse.julialang.org/t/convert-a-3d-array-to-2d-array-of-vectors/65755/8 "2021-08-04T13:34:45Z")

</div>

> [@piever](#):
>
> which IIUC under the hood uses `reinterpret`

This is correct. It applies this to something like a `PermutedDimsArray` when given the `N x K x 2` format, I’m somewhat surprised this works in fact, and would not expect it to be particularly fast. The simple re-interpretation is things like `@cast out3[j,k]{i} := vol3[i,j,k]`, making SVectors from the first dimension.

I 'm not sure it knows about ` reinterpret(reshape, ...` (which was new in Julia 1.6), so it inserts more reshapes than ideal:

```julia
julia> @cast out2[i,j]{k} := vol[i,j,k]; summary(out2)
"1000×1000 reshape(reinterpret(StaticArrays.SVector{2, Float64}, reshape(transmute(::Array{Float64, 3}, (3, 1, 2)), 2000000)), 1000, 1000) with eltype StaticArrays.SVector{2, Float64}"

```