Control flow of shell command Pipe()

General Usage
1

Hi, I am trying to process a large amount of shell command output that I want to do it line by line. The following code is what I have right now. The while loop will not terminate. I could do that by checking process_running(p). But I think then the last piece of content in the Pipe() needs to be taken care outside the while loop? Are there better way to do the job? Thank you for helping.

p = Pipe()
proc = run(pipeline(cmd, stderr=p), wait=false)
# filter out lines start with Warning
while !eof(p) #does not stop
    line = readline(p) |> String
    if ~startswith(line, "Warning")
        println(line)
    end
end
close(p.in)
close(p)
2

You need to connect the pipe to the process being executed. Assuming you want to read the process’ stderr:

p = Pipe()
proc = run(pipeline(cmd, stderr = p), wait=false)
close(p.in)
Base.start_reading(p.out)
while ....
1 Like
3

Thank you. Sorry I forgot to type =p. I corrected the post question. Your code works! Could you explain a bit what close(p.in) do? Will that block proc to put content into p?

4

I’m not sure why it’s needed, since it’s not documented. I just picked up this knowledge along the way. See https://github.com/JuliaLang/julia/issues/11824 and Improve documentation for process interaction using pipes · Issue #24810 · JuliaLang/julia · GitHub.

5

Thank you very much. Now it is able to evaluate eof(p). Do you happen to know that if the Pipe p reaches eof, does it also mean the proc is ended? So I don’t have to do wait(proc)? (In my case, process_exited(proc) returns true)

6

I’m not sure, but I think process_exited() is the most reliable way to find if the process is running or not.