Stdout buffering

General Usage
1

I seem to fail to RTFM with stream buffering.

My problem is the following (under linux):

$ strace -o trace stdbuf -o 1MB julia -e "for i=1:100000 write(stdout,\"something\") end"| cat >/dev/null
$ wc -l trace 
301126 trace

So this clearly failed to buffer.

Can anyone help me with this? My questions would be:

  1. Am I doing something wrong?
  2. How do I get a buffered stream?
  3. Where is this in the documentation? Such that I next time know where to look

If julia doesn’t respect the OS settings for buffering, then it’s also OK if I can change the stdout buffering from within julia.

PS. I believe the relevant documentation belongs into

help?> stdout
search: stdout redirect_stdout

  stdout::IO

  Global variable referring to the standard out stream.

The documentation should explain:

  1. How does the way of starting julia affect the concrete type of stdout? (tty, pipe, file)
  2. How is the buffering determined from outside julia? (environment variables, cmd flags, tools like stdbuf and if necessary analogues on bsd + macos + windows)
  3. How can you check and change buffering mode from within julia?
2 Likes
2

See:

and this thread:

maybe of help, I tried to understand the issues, and make a PR, I probably need to reread (what I wrote), posting in case helpful.

3

Found it:

$ strace -o trace julia -e "Base.buffer_writes(stdout, 1<<20); for i=1:100000 write(stdout,\"something\") end"| cat >/dev/null
$ wc -l trace 
1095 trace

The way I found it was:

  1. First, figure out what I’m dealing with: $ julia -e "@show stdout"|cat stdout = Base.PipeEndpoint(RawFD(12) open, 0 bytes waiting)
  2. Read all uses of PipeEndPoint in Base. Nothing :frowning:
  3. Read uses of Base.LibuvStream in Base until I found paydirt.

And yet,

help?> Base.buffer_writes
  No documentation found.
3 Likes