Write to file and stdout

General Usage
,
1

I have a number of println statements in a script. I know I can also do println(io,blah) to write to a file. When I do that, output no longer goes to stdout. Can I do both at once?

2 Likes
2

maybe just do it twice?

function myprintln(io, blah)
    println(blah)
    println(io, blah)
end
2 Likes
3

This needs a lot more work to define a proper IO interface, but

struct Tee{TIO <: Tuple} <: IO
    streams::TIO
end

Tee(streams...) = Tee(streams)

function _do_tee(tee, f, xs...)
    for io in tee.streams
        f(io, xs...)
    end
end

Base.write(tee::Tee, x) = _do_tee(tee, write, x)
Base.write(tee::Tee, x::Union{SubString{String},String}) = _do_tee(tee, write, x)

print(Tee(stdout, stderr), "will appear twice")
10 Likes
4

You may also consider using Logging module, it looks like it’s more natural. You can combine it with Tamas solution to simplify logger setup, and get something like

@info <some piece of code>

in a very customizable way (even with the possibility to turn it off completely).

5

Not sure whether the above package is still runnable.

6

Okay its not working.

7

Thanks for the suggestions everybody. I tried doing as @jling suggested which would have probably been fine in most cases. But I only need to do this if a boolean flag is set to true, which made that way a bit complicated and RegressionTables.jl also doesn’t work as nicely with this method.

What I ended up doing is having a single IOBuffer that all output is written to. After each section of my code, I check if the LOG_RESULTS is set to true and if it is then I additionally take from the IOBuffer to write to the output file like so

if LOG_RESULTS
    println(logfile, String(take!(copy(myio))))
end
println(String(take!(myio)))

You could actually I guess just do this once at the end of the program but in my case I like having the results print as they are available so I do that at the end of each section.

8

The package LoggingExtras might help too (in particular, there’s a TeeLogger that looks relevant, although I haven’t used it).

1 Like
9

Is there any references out there on what other methods need to be implemented for a proper IO subtype?

Or maybe more importantly, under what circumstances would you expect this implementation to be insufficient?

10

There is no formal spec as far as I can tell, but the manual should be informative. Basic write may be fine, it is just that the implementation above is not heavily tested. FWIW, I can’t even recall why I defined two methods for Base.write.

1 Like
11

One use case where the above doesn’t work: using Tee as the output stream in run(pipeline(command, tee))

I’ve tried some patches*, but couldn’t get it to work yet

*namely

  • adding Vector{UInt8} to the type union here:
  • defining
Base.write(to::Tee, from::IO) = invoke(write, Tuple{IO,IO}, to, from)
1 Like
12

(For reference and for anyone stumbling upon this: I solved my task – namely to capture a subprocess’s output but also to live-print it to stdout; not via a Tee but as follows:

buf = IOBuffer()
pos = 0
process = run(pipeline(cmd, buf), wait=false)
while process_running(process)
    sleep(0.1)
    seek(buf, pos)
    new = read(buf, String)
    print(new)
    pos += sizeof(new)
end
output = String(take!(buf))

)

4 Likes
13

You can also use Base.BufferStream for this, which is a fully implemented IO type (and it might be public API at some point).

function tee_io(captures...)
	bs = Base.BufferStream()

	t = @async begin
		while !eof(bs)
			data = readavailable(bs)
			isempty(data) && continue

			for s in captures
				write(s, data)
			end
		end
	end

	function closeme()
		close(bs)
		wait(t)
	end

	return (io=bs, close=closeme)
end

Example use, tee’ing the same data into stdout, stderr and into a buffer (created by Base.sprint):

julia> sprint() do buffer
               
               tee = tee_io(stdout, stderr, buffer)

               # use the tee
               println(tee.io, "Hello!")
               show(tee.io, "text/plain", rand(4))
               println(tee.io)

               # close it
               tee.close()
       end
Hello!
4-element Vector{Float64}:
 0.6197851577079866
 0.11502872199346159
 0.09949018487645467
 0.35495614153953925
Hello!
4-element Vector{Float64}:
 0.6197851577079866
 0.11502872199346159
 0.09949018487645467
 0.35495614153953925
"Hello!\n4-element Vector{Float64}:\n 0.6197851577079866\n 0.11502872199346159\n 0.09949018487645467\n 0.35495614153953925\n"
5 Likes