# Very slow readdlm()

**URL:** <https://discourse.julialang.org/t/very-slow-readdlm/14994>\
**Category:** General Usage\
**Created:** [September 15, 2018, 2:46pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994 "2018-09-15T14:46:49Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [September 15, 2018, 2:46pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/1 "2018-09-15T14:46:49Z")

</div>

Julia 0.6 file 500MB (only digits, linux stnadard ) on Win by readcsv() is reading in ±150 sek. Julia 0.7 has only readdlm(), The same file was reading over 2 h! What to do ? Is any package to conver file linux\>win ? It is big back step for Win useres .  
Paul

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 15, 2018, 2:47pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/2 "2018-09-15T14:47:54Z")

</div>

[https://github.com/JuliaLang/julia/issues/29036](https://github.com/JuliaLang/julia/issues/29036)

Fix will be in 1.0.1.

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [September 15, 2018, 2:49pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/3 "2018-09-15T14:49:20Z")

</div>

Nice, THX! Paul

---

<div class="post-metadata">

**Author:** ![LeoK987](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leok987/32/4880_2.png) [@LeoK987](https://discourse.julialang.org/u/LeoK987)\
**Post date:** [September 15, 2018, 11:46pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/4 "2018-09-15T23:46:42Z")

</div>

Haven’t tried to use the CSV package?

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [September 16, 2018, 6:04pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/5 "2018-09-16T18:04:32Z")

</div>

> [@LeoK987](#):
>
> CSV package

Thanks, is one step mre , convert DataFrame to Array, but now it is ok 🙂

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [October 2, 2018, 9:38am UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/6 "2018-10-02T09:38:41Z")

</div>

On 1.0.1 no better 😕  
Julia 0.6.2  
julia\> @time d=readdlm(“plik.txt”,‘,’)  
8.264069 seconds (9.32 M allocations: 347.220 MiB …

Julia 1.0.1  
julia\> @time d=readdlm(“plik.txt”,‘,’)  
54.991627 seconds (11.14 M allocations: 436.138 MiB

size of plik.txt = 56 456 KB

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 2, 2018, 1:34pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/7 "2018-10-02T13:34:32Z")

</div>

It would be good if you could provide a file which shows the slowdown.

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [October 2, 2018, 3:48pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/8 "2018-10-02T15:48:06Z")

</div>

> **[plik.rar](https://drive.google.com/file/d/1gom3HKeHZYEWQwz4nQXu-xO44cOQAsqE/view?usp=drive_open)**
>
> Google Drive file.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 2, 2018, 3:54pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/9 "2018-10-02T15:54:11Z")

</div>

Are you sure you upgraded to 1.0.1?

I get:

1.0.1

```julia
julia> @time d=readdlm("plik.txt",',');
  1.997139 seconds (9.00 M allocations: 330.061 MiB, 11.14% gc time)

julia> versioninfo()
Julia Version 1.0.1

```

0.6

```julia
julia> @time d=readdlm("plik.txt",',');
  2.057267 seconds (9.01 M allocations: 330.109 MiB, 5.33% gc time)

julia> versioninfo()
Julia Version 0.6.5-pre.0

```

---

<div class="post-metadata">

**Author:** ![programista](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/programista/32/5372_2.png) [@programista](https://discourse.julialang.org/u/programista)\
**Post date:** [October 2, 2018, 3:59pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/10 "2018-10-02T15:59:23Z")

</div>

THX,  
Yes. I am shure, but now is OK, about 7 sec. Propably another Task on my machine was lunched, sorry 🙂  
Paul

---

<div class="post-metadata">

**Author:** ![Ajaychat3](https://avatars.discourse-cdn.com/v4/letter/a/ecd19e/32.png) [@Ajaychat3](https://discourse.julialang.org/u/Ajaychat3)\
**Post date:** [October 2, 2018, 6:53pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/11 "2018-10-02T18:53:36Z")

</div>

> [@programista](#):
>
> size of plik.txt = 56 456 KB

@kristoffer.carlsson: I don’t understand one thing. The file size is about 56mb but the memory allocation is 330mb. Curious to know the reason if possible.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 2, 2018, 7:02pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/12 "2018-10-02T19:02:11Z")

</div>

The memory allocation is the total amount allocated during the execution of the function, not the memory usage when the function returns.

---

<div class="post-metadata">

**Author:** ![Ajaychat3](https://avatars.discourse-cdn.com/v4/letter/a/ecd19e/32.png) [@Ajaychat3](https://discourse.julialang.org/u/Ajaychat3)\
**Post date:** [October 2, 2018, 7:09pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/13 "2018-10-02T19:09:53Z")

</div>

So it means 330mb was allocated for reading the file. Or it is the maximum memory being allocated to entire Julia session when the file reading was in progress. Sorry if this appears to be a basic question.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [October 2, 2018, 7:31pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/14 "2018-10-02T19:31:34Z")

</div>

It means that if you sum all allocations that happened during the reading of the file, you get 330 MB.

---

<div class="post-metadata">

**Author:** ![Ajaychat3](https://avatars.discourse-cdn.com/v4/letter/a/ecd19e/32.png) [@Ajaychat3](https://discourse.julialang.org/u/Ajaychat3)\
**Post date:** [October 2, 2018, 7:33pm UTC](https://discourse.julialang.org/t/very-slow-readdlm/14994/15 "2018-10-02T19:33:09Z")

</div>

Thanks