# Freeze when reading from output of external command

**URL:** <https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034>\
**Category:** General Usage\
**Created:** [February 13, 2018, 1:54pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034 "2018-02-13T13:54:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeffersyno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffersyno/32/3430_2.png) [@Jeffersyno](https://discourse.julialang.org/u/Jeffersyno)\
**Post date:** [February 13, 2018, 1:54pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034/1 "2018-02-13T13:54:11Z")

</div>

Hi, I’m having an issue with running an external command from Julia. I’m trying to read XML from a process’s output stream and parse it with LibExpat. The problem I’m having is that Julia freezes indefinitely at the first run. When I Ctrl+C (staying in the same REPL instance) and run again, it works as expected. I’m able to reproduce the problem with the following code snippet:

```julia
open(`cat about-500kb-of-xml.osm`, "r") do io
    println(readline(io)) # should just print the first line of the xml file, but freezes at the first run, works fine the second time
end

```

When it’s frozen, I can see the `cat` process as a subprocess of Julia. The file size seems to matter; I don’t experience any problems with smaller files. The stacktrace when Ctrl+C’ing is:

```julia
 [1] stream_wait(::Base.PipeEndpoint, ::Condition, ::Vararg{Condition,N} where N) at ./stream.jl:42
 [2] close(::Base.PipeEndpoint) at ./stream.jl:326
 [3] open(::Cmd, ::String, ::Base.DevNullStream) at ./process.jl:579
 [4] open(::Cmd, ::String) at ./process.jl:575
 [5] open(::##3#4, ::Cmd, ::String) at ./process.jl:599

```

Any idea what might be causing the freeze? Julia version 0.6.2. Thanks 🙂

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [February 13, 2018, 3:26pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034/2 "2018-02-13T15:26:12Z")

</div>

Is `cat` just a stand in for a more complex command here? If it’s not, then you should just open the file itself for reading, as in

```julia
open("about-500kb-of-xml.osm", "r") do io
    println(readline(io))
end

```

Julia output is not line buffered, so if you want to flush the output after each line, you’ll need an explicit call to `flush(STDOUT)`. It’s possible that `cat`’s (or whatever command’s) output buffering is conflicting with Julia’s input buffering somehow, but it’s hard to say. Try it without the unnecessary `cat` process and/or with the `flush` call and see which if either helps.

---

<div class="post-metadata">

**Author:** ![Jeffersyno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffersyno/32/3430_2.png) [@Jeffersyno](https://discourse.julialang.org/u/Jeffersyno)\
**Post date:** [February 13, 2018, 3:44pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034/3 "2018-02-13T15:44:52Z")

</div>

`cat` is indeed a stand-in for a more complex command. The command itself converts binary-encoded OpenStreetMap data into XML. I used `cat` to provide an easily (assumably) reproducible example.

Everything works as expected when directly reading from a file.

I’d like to avoid creating the file explicitly, though, as the XML files easily grow a few 100MBs in size.

With respect to the buffering, the problem occurs regardless of what I do in the do-expression. I used a print statement for simplicity. The freeze seems to occur even before the do-expression is compiled (JIT magic).

---

<div class="post-metadata">

**Author:** ![jameson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jameson/32/23_2.png) [@jameson](https://discourse.julialang.org/u/jameson)\
**Post date:** [February 13, 2018, 4:40pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034/4 "2018-02-13T16:40:25Z")

</div>

I thought we back ported the fix for this ([https://github.com/JuliaLang/julia/pull/22886](https://github.com/JuliaLang/julia/pull/22886)), but it looks like it was missed when making v0.6.1.

---

<div class="post-metadata">

**Author:** ![Jeffersyno](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffersyno/32/3430_2.png) [@Jeffersyno](https://discourse.julialang.org/u/Jeffersyno)\
**Post date:** [February 13, 2018, 8:24pm UTC](https://discourse.julialang.org/t/freeze-when-reading-from-output-of-external-command/9034/5 "2018-02-13T20:24:08Z")

</div>

Issue does indeed not occur in Julia 0.7.0-DEV.396. Thanks for the info 🙂