I have an interactive python script that writes to and asks for input from the terminal multiple times. Is there an easy way to interact with it? I tried to redirect input and output through Pipe and PipeBuffer but couldn’t figure it out. I also found this but I can’t extend this for multiple I/O.
Hi @Machine-Teacher,
it would probably helpful if you would post some code excerpt to help us reason about it.
Thanks!
Hi @goerch,
I would like my code to work like this. This is obviously not working code, but perhaps someone can suggest how to implement similar functionality.
function communicate(cmd::Cmd)
i, o, e = Pipe(), Pipe(), Pipe()
process = run(pipeline(cmd; stdin=i, stdout=o, stderr=e); wait=false)
close(o.in)
close(e.in)
while true
stdout = @async String(read(o))
stderr = @async String(read(e))
println(String(fetch(stdout)))
write(i, readline())
fetch(e) != 0 && break
wait(o)
end
println(fetch(e))
kill(process)
end
Thanks for posting. I’m by no means an expert in this but based on your code I tried something like
function communicate(cmd::Cmd)
i, o, e = Pipe(), Pipe(), Pipe()
process = run(pipeline(cmd; stdin=i, stdout=o, stderr=e); wait=false)
try
Threads.@threads for stream in [stdin, stdout, stderr]
if stream == stdin
try
while true
s = readline()
println(i, s)
end
catch exception
!(exception isa InterruptException) && rethrow()
end
elseif stream == stdout
while true
s = String(readline(o))
println(s)
end
elseif stream == stderr
while true
s = String(readline(e))
println(s)
end
end
end
catch exception
!(exception isa InterruptException) && rethrow()
finally
kill(process)
end
end
communicate(Cmd(["C:/Users/Win10/AppData/Local/Programs/Julia-1.8.0-DEV/bin/julia.exe"]))
Maybe this gets the discussion started. Should this be done with Threads or Tasks?
Thanks for your reply. I believe this would be better done with sequential tasks. After all, the processes (reading, writing) are sequential, I don’t understand why threads should be used for this. In addition, in the general case, the program should continue until the end of the process, but I just can’t implement it.
This is an assumption isn’t it? And in general we don’t know the exact response of the communication partner (for example how many lines (bytes) of response and when to expect them).
I believe tear down handling is a special case (I just checked sigint). Which process will end first, parent or child?
Yes it is. In general, we don’t know the IO order, but this seems to me a bit more cluttered solution.
In the general case, it can be different, but in my case a child process will end first.
I just checked @tkf 's original code: this seems to solve the communication problem for one shot execution. I also don’t see how to extend this to fulfill your requirement
In absence of other information I believe Threads is the way to go.
Using Threads.@threads for is not the right tool for this. You’d need to wrap blocking call in a Threads.@spawn or @async.
1 Like
Thanks! Something like this
function communicate(cmd::Cmd)
i, o, e = Pipe(), Pipe(), Pipe()
process = run(pipeline(cmd; stdin=i, stdout=o, stderr=e); wait=false)
try
@async (
while true
s = String(readline(o))
println(s)
end
)
@async (
while true
s = String(readline(e))
println(s)
end
)
while true
s = readline()
println(i, s)
end
catch exception
!(exception isa InterruptException) && rethrow()
finally
kill(process)
end
end
communicate(Cmd(["C:/Users/Win10/AppData/Local/Programs/Julia-1.8.0-DEV/bin/julia.exe"]))
then?
Yeah, I think you got the idea. But a few more comments:
If you don’t have @sync around @async, that’s usually a bad idea. The errors in @async tasks would be ignored and it’d be hard to debug.
I think you to close the output pipes o.in and e.in after run if you want to stop the tasks when the program exit.
The while loops can be written as for line in eachline(o) ... and for line in eachline(stdin) ....
1 Like
Thanks again. I tried to incorporate your suggestions:
function communicate(cmd::Cmd)
i, o, e = Pipe(), Pipe(), Pipe()
process = run(pipeline(cmd; stdin=i, stdout=o, stderr=e); wait=false)
try
@sync for s in eachline()
@async (
for s in eachline(o)
println(s)
end
)
@async (
for s in eachline(e)
println(s)
end
)
println(i, s)
end
catch exception
!(exception isa InterruptException) && rethrow()
finally
kill(process)
close(o.in)
close(e.in)
end
end
communicate(Cmd(["C:/Users/Win10/AppData/Local/Programs/Julia-1.8.0-DEV/bin/julia.exe"]))
You don’t want to schedule tasks for each line. Also, the input endpoint should be closed before waiting for the “reader” tasks (that automatically happens at the end of the @sync block). I think you’d need something like this (untested):
function communicate(cmd::Cmd)
i, o, e = Pipe(), Pipe(), Pipe()
process = run(pipeline(cmd; stdin=i, stdout=o, stderr=e); wait=false)
close(o.in)
close(e.in)
@sync try
@async for s in eachline(o)
println(s)
end
@async for s in eachline(e)
println(s)
end
for s in eachline()
println(i, s)
end
catch exception
!(exception isa InterruptException) && rethrow()
finally
kill(process)
end
end
1 Like
Thanks, it is more closer. I tested your solution and after the process ends it gives all the output in one go. I would like to get the current output while the script is still running. And is it really impossible to end the script without directly interrupting it?
Refactor your script so that you can call it as a library, by calling functions (via PyCall in Julia) rather than I/O. Don’t write scripts for code that you want to re-use, write functions/modules/classes.
2 Likes
Is it really that bad?
Yes. (This is regardless of what language you are using.)
2 Likes