Communicate with external process through pipe

General Usage
1

I am trying to create a pipe (a pair of file descriptors), find the file descriptor number of the writing end of the pipe, pass it to an external command and read from the reading end of the pipe from my Julia script. In short, I need to create a pipe, run an external command and make sure that it inherits the appropriate file descriptor from my script. From Running External Programs, I’ve learned about Cmd and run and from this answer about the Pipe type which I am unable to locate using the search bar at docs.julialang.org. I tried running

p = Pipe()
println(p)

from a script that came back with a stack trace which I was able to use to locate the definitions of the PipeEndpoint and Pipe types. From here, I could start reading the source but I figured I shoud double-check first whether I’m on the right track at all.

EDIT:

This is what I’m trying to do in Python:

import os
import subprocess

r, w = os.pipe()

command = ["perf", "stat"] + ["--log-fd",
                              f"{w}"] + ["dd", "if=/dev/zero", "of=/dev/null", "count=1000000"]

popen = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE, pass_fds=[w], text=True)

stdout, stderr = popen.communicate()
popen.wait()

os.close(w)
file_object = open(r)
stats = file_object.read()
file_object.close()

print(f"stdout:\n{stdout}")
print(f"stderr:\n{stderr}")
print(f"stats:\n{stats}")

In Julia, I tried using a simple file instead of a pipe but it seems that the file descriptor is not kept open between the parent and child:

pstdout = Pipe()
pstderr = Pipe()

io = open("stats", "w+")
filedescriptor = fd(io)

proc = run(pipeline(`perf stat --log-fd $filedescriptor dd if=/dev/zero of=/dev/null count=1000000`, stdout = pstdout, stderr = pstderr), wait=false)

close(pstdout.in)
close(pstderr.in)

stdout = String(read(pstdout))
stderr = String(read(pstderr))

println("stdout:")
println(stdout)

println("stderr:")
println(stderr)
2

I’m not sure if this is what you need, but this is what I do to communicate with an external process using pipes:

pstdin = Pipe()
pstdout = Pipe()
pstderr = Pipe()
proc = run(pipeline(`externalprogram`,
                    stdin = pstdin, stdout = pstdout, stderr = pstderr),
                    wait = false)
process_running(proc) || error("There was a problem.")
close(pstdout.in)
close(pstderr.in)
close(pstdin.out)

At this point, you can communicate with the external program by writing to pstdin, and reading from pstdout and pstderr.

1 Like
3

Thank you for your reply! Communicating through stdin, stdout and stderr is only half of the story… I edited my question to make it clearer what I am after.