Function call on STDOUT

Hello everyone.

Let me describe my situation.
In Term I’m working on a progress bar feature.
It looks like this:

The way the bar display is updated is by using ANSI codes to erase the last line printed out in the terminal before printing out the new bar. Just printing out the bar would cause this:

The idea is that you can use the progress bar to measure progress through a loop. Something like:

pbar = ProgressBar(100)
for i in 1:100
   # do something

My problem is that any call print or println will disrupt my visualization. Since the whole idea is to erase the last line in the console before printing the updated progress bar, print won’t show anything because its output will be erased and println will cause the progress bar to show up on a new line.

So here’s the main problem. If I can keep track of any call to print and println, I can determine on which line the progress bar should be and only erase that, while still showing all stdout output correctly. So what I would like is to have a function that gets called any time an stdout output producing function is called, so that I can keep track of how many lines have been printed while the progress bar was being displayed. The question is: how to do that?.

I know redirect_stdout exists, and has a method that accepts a function as argument. But that function gets called only once and then stdout is restored. I want a way to have the normal stdout, but have my function called everytime something gets printed. Is there a way to do something like that?

Thank you,

That’s not a place to “hook into” - you’d have to hook extremely early in running any code, redirect_stdout for all subsequent calls in a “regular” session. I’m not sure this is feasible globally with the way the whole system works right now.

Fundamentally, this is an issue with how file descriptors & printing works. All redirect_stdout does under the hood is swap out the stdout file descriptor/object for a custom one to print to. There’s no facility to do this globally & permanently.

thanks for the reply.

It doesn’t have to be a global/permanent solution. Essentially it’s just for the duration of the for loop in which the progress bar gets updated.

I suppose one way would be to re-define Base.print, but I would rather avoid that. I’m not very familiar with IO objects in Julia, but would there be a way to create a custom buffered IO so that I can monitor what goes in/out of it? And then redirect stdout to work with that for the duration of the loop?

If it’s only for the duration of a for loop, how about using the do notation to encapsulate the loop?

Like this:

redirect_stdout() do
   # loop goes here

You could of course also use the excellent ProgressBars.jl, though since you’re writing your own, this may not be what you want.

Thanks. This would required the user encapsulating all their loops with the appropriate code.
I need Term to handle it behind the scene: re-direct the stdout when the progress bar is created and restore it when it’s all done.

I’m lookingo using a custom IOBuffer or IOStream, as soon as I figure out how they work! Thanks for the suggestion, I’ll see how ProgressBars does things, maybe that’ll help!

Edit: a quick test showed that ProgressBars suffers from the same problems described in my original post.

Like I said, it’s a fundamental limitation. There’s nothing you can really do about it without more or less rewriting how this works in julia itself. You’d basically have to abstract printing away from how it works right now and towards having a single/central bottleneck that everything has to go through, allowing you to specify a “callback” of sorts to register from the user side. More or less keeping some “hidden state” between some special invocations in some arbitrary code. This just isn’t supported and I think the likelyhood of it being supported in the future is rather low. This would also make julia rather different from how this works in (to my knowledge) almost every programming language out there.

Another complication (you may not have thought of yet) is how this would/should interact with multithreaded code - you have all the same problems, except now you also have to deal with locking the stdout object, dealing with race conditions there, getting the same problems you possibly already dealt with in user code but now keeping in mind that other code may have already released its “progress bar” lock etc…

In short, in some capacity the user has to be aware of how printing works to not run into this.

1 Like

I see. That seems rather complicated indeed.

I guess I will leave it as it is for now and just warn users of this behavior. Thanks for the help!

1 Like

@Sukera for reference, I think this mostly does what I want without having to go through intercepting calls to print directly. The code is based on the awesome Suppressors.jl package.

original_stdout = stdout
out_rd, out_wr = redirect_stdout()
out_reader = @async read(out_rd, String)

for i in 1:5
    println("show later")
    println(original_stdout, "shownow")


out = fetch(out_reader)


With output

show later
show later
show later
show later
show later

The idea is that any call like print("text") will not show until I’ve restored the original STDOUT stream, but I can use println(original_stdout, "text") display things immediately. So the way it’s going to play out is that when the progress bar is created I re-direct the stdout:

original_stdout = stdout
out_rd, out_wr = redirect_stdout()
out_reader = @async read(out_rd, String)

Then during the loop the user can use print as they do normally, but the output won’t show immediately.

When the loop is done and the progress bar stopped, it can restore the normal stdout behavior and print out anything that was captured by the re-directed stream.

Well to me that just reads like using the do notation, but with extra steps :thinking: Would be a good use case for a macro, though I’m not sure delaying printing in case of an infinite loop works quite as well… Seems niche :person_shrugging:

I see what you’re saying, but it will work out differently from what you think.

Term will take care of all the stdout handling, so for the user it will all just magically work. This is still work in progress, but the idea is to re-direct the stdout when the progress bar starts and restored it when it stops.

What that means is that this code:

using Term

N = 100
pbar = ProgressBar(;N=N)

for i in 1:N
    if i % 25 == 0

results in this output:

Here out is not printed out until after the loop is completed so that it doesn’t disrupt the bar’s visualization.

In summary, the user doesn’t have to change their code or know how to handle STDOUT etc. Term does it for them.

You may have a look at how TerminalLoggers.jl does it, for example the following does what you want:

julia> using Logging, ProgressLogging, TerminalLoggers

julia> global_logger(TerminalLogger());

julia> @progress for i in 1:10
           println("hello from step $i")


The solution I’ve described above works well for me, but I’d like to see other implementations. I can’t see where @progress is defined though?