# What is the most efficiency way to read the 3D matrix?

**URL:** https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184
**Category:** Performance
**Tags:** io
**Created:** [April 6, 2023, 3:29pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184 "2023-04-06T15:29:30Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![manu\_han](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manu_han/32/47520_2.png) [@manu\_han](https://discourse.julialang.org/u/manu_han)
#### Post date: [April 6, 2023, 3:29pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/1 "2023-04-06T15:29:30Z")

</div>

Here is 3 D array.  
TD=rand(150,27,55)

I have saved ‘TD’ in a various way as follow.

1. DataFrame(λ\_SPS=collect(eachslice(λ\_SPS,dims=3)))  
CSV.write(“TD.csv”, TD)
2. DataFrame(λ\_SPS=collect(eachslice(λ\_SPS,dims=1)))  
CSV.write(“TD.csv”, TD)
3. writedlm(“TD.txt”, TD)

The memory and speed of the three methods are as follows. (using @btime CSV.read(“data”,df))

1. 24.28 MiB, 85.586 ms
2. 15.03 MiB, 48.067 ms
3. 1.87 MiB, 22.547 ms

Results:

1. Reading the .txt using ‘CSV.read’, the process was most efficiency. (readdlm was terrible)
2. As down the size of the matrix in dataframe, process speed was getting better. the number of row was not important comparing to size of the matrix.

I tested various method, but i couldn’t find the best way, yet. please share your experience.

---

<div class="post-metadata">

### Author: ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [April 6, 2023, 3:43pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/2 "2023-04-06T15:43:03Z")

</div>

You can dump the array to disk using JLD.jl or JLD2.jl, you don’t need to convert to a table and write as csv. There are other alternative packages like BSON.jl etc that you can use to store arrays.

---

<div class="post-metadata">

### Author: ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [April 6, 2023, 6:41pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/3 "2023-04-06T18:41:10Z")

</div>

Julia standard library’s Serialization takes 2 ms to write, less than 0.5 ms to read and fills 1.7 MB on disk, on my small Win11 laptop, but not sure if the experts would recommend it:

```julia
using Serialization
TD = rand(150,27,55)
serialize("TD_serialize.bin", TD); # 2.0 ms (24 allocs: 1.8 KiB)
TD2 = deserialize("TD_serialize.bin") # 463 μs (31 allocs: 1.7 MiB)
TD == TD2 # true

```

---

<div class="post-metadata">

### Author: ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [April 8, 2023, 1:15pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/4 "2023-04-08T13:15:10Z")

</div>

Very nice @rafael.guerra , thanks for sharing this stdlib, I will consider it next time. 🙂

---

<div class="post-metadata">

### Author: ![magister-ludi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/magister-ludi/32/4003_2.png) [@magister-ludi](https://discourse.julialang.org/u/magister-ludi)
#### Post date: [April 8, 2023, 1:31pm UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/5 "2023-04-08T13:31:23Z")

</div>

It’s worth remembering the Julia documentation:  
`The data format can change in minor (1.x) Julia releases, but files written by prior 1.x versions will remain readable.`  
In many contexts, this won’t be an issue, but it should be kept in mind.

---

<div class="post-metadata">

### Author: ![manu\_han](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manu_han/32/47520_2.png) [@manu\_han](https://discourse.julialang.org/u/manu_han)
#### Post date: [April 12, 2023, 2:01am UTC](https://discourse.julialang.org/t/what-is-the-most-efficiency-way-to-read-the-3d-matrix/97184/6 "2023-04-12T02:01:40Z")

</div>

Oh, I forgot wirte the thanks comments.  
It was really helpful and improve dataread performance 200 times!  
Thank you Rafael.
