Document process interaction with stdin, stout, stderr

Now that the 1.0 feature freeze is on the horizon, I’d like to re-surface a long-standing issue and make a plea for it to be addressed in 1.0.

Sometimes one wants to launch a long-living external program from Julia, and iteract with it via STDIN, STDOUT and STDERR. Note that direct access to the pipes is needed; it’s not enough to connect the pipes to files.

There is still no documented way to do this, although it is possible using undocumented functions, as I do here.

I initially raised this issue back in 2015; it has also been discussed here.

From the standpoint of a Julia programmer, having this functionality added to standard commands such as readandwrite would be ideal. A close second best would be to officially document and sanction a way to do this. My concern with relying on undocumented features is that code such as my popen3 function may be broken any time Julia’s internal API changes.

If there is consensus that this is something that should be addressed, I’ll open a new issue.

– mb

2 Likes

I’m not really sure what you’re saying here. The supported way to get all three streams as pipes is the following

julia> stdin, stdout, stderr = (Pipe() for _ in 1:3);

julia> spawn(`cat`, stdin, stdout, stderr) # or equivalently spawn(pipeline(`cat`, stdin=stdin, stdout=stdout, stderr=stderr))
Process(`cat`, ProcessRunning)

The readandwrite function is mostly a convenience API for the common case of needing only stdin/stderr.
Though, since we made processes readable/writable, we should probably change it to just return a process object.

Are you looking for better documentation?

Looking at this again, readandwrite was meant to be deprecated, there’s even a TODO left in the code to do so. PR incoming. Also, upon further consideration, the easiest way to do this (though less symmetric with the other streams) is:

open(pipeline(`cat`; stderr=Pipe()), "r+")
# Use p.{in,out,err} to access

This did have me looking back at some of this code and I found a bug. Issue filed: https://github.com/JuliaLang/julia/issues/24717

Yes. Neither Pipe() nor that use of spawn are documented.

Ok, please feel free to file a doc issue. Maybe we should have a tag or milestone for doc issues that are addressable within the 1.0 timeframe. Would be good to take some time for docing after the feature freeze.

1 Like

Yes, please!

Done: Improve documentation for process interaction using pipes · Issue #24810 · JuliaLang/julia · GitHub

I hope this issue can be added to the 1.0 milestone list.

@Keno Is this supposed to work in 0.6? It doesn’t work for me.

julia> p=open(pipeline(`gnuplot`; stderr=Pipe()), "r") # Note the "r"
(Pipe(RawFD(-1) closed => RawFD(20) open, 0 bytes waiting), Process(`gnuplot`, ProcessExited(0)))

No, I believe that change happened in the 0.7 timeframe.

OK, thanks! I’ll try it out when I have a chance to build 0.7.