This is not nice at all!
Simple reproduction:
julia> pp=open(`head -n 1`, "r+")
Process(`head -n 1`, ProcessRunning)
julia> write(pp, 1);
julia> write(pp, 2);
julia> write(pp, 3);
julia> write(pp, "\n");
julia> write(pp, "\n");
ERROR: IOError: write: broken pipe (EPIPE)
Expectation: Only send the data and error once the pipe buffer is full or closed or manually flushed (buffer might mean /proc/sys/fs/pipe-max-size bytes, which is 1 MB for me).
Alternatively, create and document options for buffering (options: no buffering, until newline, until full with option to shrink below system max). I would have expected a discussion of all available options in https://docs.julialang.org/en/v1/devdocs/stdio/ and/or https://docs.julialang.org/en/v1/manual/running-external-programs/ and/or https://docs.julialang.org/en/v1/manual/networking-and-streams/.
If these options exist, it would be your job to check whether stdin/stdout/stderr are pipes and enable proper buffering.It would be preferable if open of commands had proper buffering options; alternatively, one should set them after the fact (first open, then set buffering options).
I don’t understand whether libuv or the sendbuf field of PipeEndpoint is supposed to be responsible for buffering. In view of all the multithreading issues of lock-or-no-lock I suspect that libuv must do the buffering and sendbuf is dead code?
Also, I have committed the same sin of passing pipes to opened commands around like candy; but if writes are unbuffered, then this is hell. Thank you for making me aware of this problem!
I am tempted to open an issue for this, but will let it wait for another day on discourse in case this is a failure-to-RTFM problem on my side. Pinging @jameson due to commit history of streams.jl.
Also, I don’t really get libuv. Very short poking to see whether there is an obvious bad option set leaves me at a loss.
julia> nn=Ref(Cint(0));
julia> ccall(:uv_recv_buffer_size, Cint, (Ptr{Cvoid},Ptr{Cint}), pp.in.in.handle, nn)
-88
julia> ccall(:uv_send_buffer_size, Cint, (Ptr{Cvoid},Ptr{Cint}), pp.in.in.handle, nn)
-88
julia> ccall(:uv_is_writable, Csize_t, (Ptr{Cvoid},), pp.in.in.handle)
0x0000000000000001
julia> ccall(:uv_stream_get_write_queue_size, Csize_t, (Ptr{Cvoid},), pp.in.in.handle)
ERROR: ccall: could not find function uv_stream_get_write_queue_size
I presume that the negative value actually means 2GB, since “misreported by factor of two on linux” which wraps negative (or this is supposed to be an unsigned int, or size_t or the libuv documentation sucks). And I found no API for checking whether a pipe is set to be blocking.
PS. Opened an issue.