# Massive \`readstring\` -\> Too many open files

**URL:** https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426
**Category:** General Usage
**Created:** [April 29, 2017, 3:19am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426 "2017-04-29T03:19:04Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ulysses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ulysses/32/665_2.png) [@ulysses](https://discourse.julialang.org/u/ulysses)
#### Post date: [April 29, 2017, 3:19am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/1 "2017-04-29T03:19:04Z")

</div>

My program processs large amount of files using `readstring`. Eventually it dies with `Too many open files`. I quess the reason is the way `read` function is implemented: it just do `open` but do not `close`, neither use `do` block. Should I report a bug? Or I misunderstand something?

---

<div class="post-metadata">

### Author: ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)
#### Post date: [April 29, 2017, 3:49am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/2 "2017-04-29T03:49:23Z")

</div>

`readstring` appears to be properly closing the file [https://github.com/JuliaLang/julia/blob/8eca02717bcea524877fad01aacedc807898bcd1/base/io.jl#L537](https://github.com/JuliaLang/julia/blob/8eca02717bcea524877fad01aacedc807898bcd1/base/io.jl#L537) by using the callback verion. Which function are you calling?

---

<div class="post-metadata">

### Author: ![ulysses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ulysses/32/665_2.png) [@ulysses](https://discourse.julialang.org/u/ulysses)
#### Post date: [April 29, 2017, 4:23am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/3 "2017-04-29T04:23:58Z")

</div>

I thought that it is `readstring` from `base/process.jl` which is called. It invokes [`read` which, AFAIS, don’t do close](https://github.com/JuliaLang/julia/blob/8eca02717bcea524877fad01aacedc807898bcd1/base/process.jl#L631).

I’m not sure how Julia decides which of two (io.jl vs. process.jl) `readstring` to call.

---

<div class="post-metadata">

### Author: ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)
#### Post date: [April 29, 2017, 5:05am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/4 "2017-04-29T05:05:31Z")

</div>

If you are using the file one then it’s not the one for `AbstractCmd`. See [the doc](https://docs.julialang.org/en/latest/manual/methods.html) for how these works. The one for command is closed after the read.

---

<div class="post-metadata">

### Author: ![ulysses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ulysses/32/665_2.png) [@ulysses](https://discourse.julialang.org/u/ulysses)
#### Post date: [April 29, 2017, 4:03pm UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/5 "2017-04-29T16:03:03Z")

</div>

Now I see. It seems, that calling `readstring("/path/to/file")` turns into the following chain of calls:

```julia
(1) readstring("/path/to/file") → 
(2) open(readstring, "/path/to/file") → 
(3) readstring(file_handle) → 
(4) read(file_handle) → ...

```

And `open` (2) do `close` handle after `readstring` (3) returns.

It stays unclear though, why many `readstring`s lead to `Too many open files`. I can’t provide minimum working example so far (the program crushed with this is huge). I’ll try to do it asap.

---

<div class="post-metadata">

### Author: ![ulysses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ulysses/32/665_2.png) [@ulysses](https://discourse.julialang.org/u/ulysses)
#### Post date: [April 29, 2017, 4:39pm UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/6 "2017-04-29T16:39:56Z")

</div>

I found the other place in the project with `open` not balanced with `close`.

---

<div class="post-metadata">

### Author: ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)
#### Post date: [April 29, 2017, 4:56pm UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/7 "2017-04-29T16:56:02Z")

</div>

For future reference, not sure what OS you are on, at least on linux it is easy to check `/proc/self/fd` and see what are the opened files, that should give you an idea of when/where it happens. I imaging windows/mac have similar tools.

---

<div class="post-metadata">

### Author: ![ulysses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ulysses/32/665_2.png) [@ulysses](https://discourse.julialang.org/u/ulysses)
#### Post date: [April 29, 2017, 5:39pm UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/8 "2017-04-29T17:39:23Z")

</div>

I’m on Linux. This will be helpful, thank you very much!

---

<div class="post-metadata">

### Author: ![jameson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jameson/32/23_2.png) [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [May 1, 2017, 5:39am UTC](https://discourse.julialang.org/t/massive-readstring-too-many-open-files/3426/9 "2017-05-01T05:39:48Z")

</div>

> [@yuyichao](#):
>
> on linux it is easy to check /proc/self/fd and see what are the opened files

`lsof` exists on many UNIX dialects (including mac) to do this in a more general manner
