# Fastest way to turn table into Array?

**URL:** <https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255>\
**Category:** Performance\
**Tags:** question\
**Created:** [May 18, 2022, 11:59am UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255 "2022-05-18T11:59:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![marianoarnaiz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marianoarnaiz/32/19377_2.png) [@marianoarnaiz](https://discourse.julialang.org/u/marianoarnaiz)\
**Post date:** [May 18, 2022, 11:59am UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/1 "2022-05-18T11:59:57Z")

</div>

Hi all.

One if the main parts of my daily workflow makes me turn large tables to matrixes and or n-dims-arrays.

For example:

For a X Y Z Property table with 1 million+ lines, I do:

```julia

Lat=-90:1:90
Long=-180:1:180

Zi=[410 471 660 760 821 921 1021 1121 1221 1321 1421 1521 1621 1721 1821 1921 2021 2121 2221 2321 2421 2521 2621 2740 2771 2889];

dRho=readdlm("ALL_dRho",Float64,comments=true,comment_char='#'); #input file

dRhoM=zeros(size(Long,1),size(Lat,1),size(Zi,2));

for a=1:size(Long,1)
    for b=1:size(Lat,1)
        for c=1:size(Zi,2)
            println("$a $b $c")
            indx=findall( (dRho[:,2].==Lat[b]) .& (dRho[:,1].==Long[a]) .& (dRho[:,3].==Zi[c]) );
            @inbounds dRhoM[a,b,c]=dRho[indx[1],4];
        end
    end
end

```

Is there any way to make this allocation faster? It takes hours as it is!

If anyone wants to try, the file ALL\_dRho.zip is here:

> [GitHub - marianoarnaiz/JULIA](https://github.com/marianoarnaiz/JULIA)

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [May 18, 2022, 12:05pm UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/2 "2022-05-18T12:05:31Z")

</div>

have your read the performance tips section of the manual? there are 2 things you could do that will make this much faster.

---

<div class="post-metadata">

**Author:** ![skleinbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skleinbo/32/36080_2.png) [@skleinbo](https://discourse.julialang.org/u/skleinbo)\
**Post date:** [May 18, 2022, 12:59pm UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/3 "2022-05-18T12:59:35Z")

</div>

Luckily, the data is sorted the right way for this to work…

```julia
dRhoM = reshape(dRho[:,4], length(Long), length(Lat), length(Zi));

```

Other than that, the print statement is a major source of slowing down. The loops are in the wrong order too. Remember, Julia lays out arrays in column-major-order.

---

<div class="post-metadata">

**Author:** ![joa-quim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joa-quim/32/227_2.png) [@joa-quim](https://discourse.julialang.org/u/joa-quim)\
**Post date:** [May 18, 2022, 3:11pm UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/4 "2022-05-18T15:11:10Z")

</div>

> [@skleinbo](#):
>
> `eachindex` is just `1:length` in this case, but more generic.

It looks not

```julia
dRhoM = reshape(dRho[:,4], 1:length(Long), eachindex(Lat), eachindex(Zi));
ERROR: MethodError: no method matching reshape(::Vector{Float64}, ::Tuple{UnitRange{Int64}, Base.OneTo{Int64}, Base.OneTo{Int64}})

```

---

<div class="post-metadata">

**Author:** ![skleinbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skleinbo/32/36080_2.png) [@skleinbo](https://discourse.julialang.org/u/skleinbo)\
**Post date:** [May 18, 2022, 3:29pm UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/5 "2022-05-18T15:29:12Z")

</div>

Ah, you are right. I think I was mentally in the wrong thread 😄 . I have edited the statement.

Yet

> It looks not

no trouble here…

```julia
julia> dRhoM = reshape(dRho[:,4], 1:length(Long), eachindex(Lat), eachindex(Zi));

julia>

```

Then I realized that `dRhoM` is an `OffsetArray` though I never imported that package. Must have extended `Base.reshape` when loading a package that depends on it. That’s logical but a bit nasty.

---

<div class="post-metadata">

**Author:** ![joa-quim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joa-quim/32/227_2.png) [@joa-quim](https://discourse.julialang.org/u/joa-quim)\
**Post date:** [May 18, 2022, 3:51pm UTC](https://discourse.julialang.org/t/fastest-way-to-turn-table-into-array/81255/6 "2022-05-18T15:51:54Z")

</div>

outch, looks like that `OffsetArray` is indeed a dangerous beast.