# A minimal example with Base.redirect\_stdout

**URL:** <https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245>\
**Category:** General Usage\
**Tags:** question\
**Created:** [July 7, 2021, 11:52pm UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245 "2021-07-07T23:52:48Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![harris-chris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/harris-chris/32/23877_2.png) [@harris-chris](https://discourse.julialang.org/u/harris-chris)\
**Post date:** [July 7, 2021, 11:52pm UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/1 "2021-07-07T23:52:49Z")

</div>

Hello all

I’m having a bit of difficulty making the `Base.redirect_stdout(f::Function, stream)` method work. What I would have thought would be a minimal working example is in fact a minimal failing example:

```julia
out = Pipe()
redirect_stdout(out) do s
    println("stick this in your pipe")
end

```

This gives me  
`ERROR: MethodError: no method matching redirect_stdout(::Pipe)`

Who’s in the wrong here - me, or Julia? Any help much appreciated. It seems like a nifty function and it’s annoying not to be able to use it.

Link to the docs below:  
[https://docs.julialang.org/en/v1/base/io-network/#Base.redirect\_stdout-Tuple{Function,%20Any}](https://docs.julialang.org/en/v1/base/io-network/#Base.redirect_stdout-Tuple%7BFunction,%20Any%7D)

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [July 8, 2021, 12:08am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/2 "2021-07-08T00:08:21Z")

</div>

Redirection to `Pipe` is not supported on Julia 1.6. You can redirect to a file stream:

```julia-repl
julia> open("/tmp/stdout", "w") do io
           redirect_stdout(io) do
               println("hello, world")
           end
       end

julia> read("/tmp/stdout", String)
"hello, world\n"

```

In the upcoming Julia 1.7 `Pipe` is supported, though ([JuliaLang/julia#39132](https://github.com/JuliaLang/julia/pull/39132)).

---

<div class="post-metadata">

**Author:** ![ess3sq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ess3sq/32/26834_2.png) [@ess3sq](https://discourse.julialang.org/u/ess3sq)\
**Post date:** [July 8, 2021, 12:10am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/3 "2021-07-08T00:10:00Z")

</div>

> [@fredrikekre](#):
>
> Rediretion to `Pipe` is not supported on Julia 1.6.

Why though? The docs very clearly state that it is:  
[https://docs.julialang.org/en/v1/base/io-network/#Base.redirect\_stdout-Tuple{Function,%20Any}](https://docs.julialang.org/en/v1/base/io-network/#Base.redirect_stdout-Tuple%7BFunction,%20Any%7D)

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [July 8, 2021, 12:12am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/4 "2021-07-08T00:12:56Z")

</div>

Bug in the documentation ([JuliaLang/julia#31159](https://github.com/JuliaLang/julia/pull/31159)).

---

<div class="post-metadata">

**Author:** ![harris-chris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/harris-chris/32/23877_2.png) [@harris-chris](https://discourse.julialang.org/u/harris-chris)\
**Post date:** [July 8, 2021, 1:00am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/5 "2021-07-08T01:00:35Z")

</div>

OK. Seems like a handy function when it’s ready.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [July 8, 2021, 9:56am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/6 "2021-07-08T09:56:00Z")

</div>

This works in Julia 1.7-beta2:

```julia
pipe = Pipe()
output = IOBuffer()
cmd = run(pipeline(pipe, `wc -l`, result), wait=false)
redirect_stdout(pipe) do
    print(pipe, "a\nb\n")
end

close(pipe)
wait(cmd)

julia> result = String(take!(output))
"2\n"

```

You can also read directly from the pipe, but don’t forget that pipe writes will block if the pipe is full (because the reader is slower), so you need something like this:

```julia
pipe = Pipe()
writer = @async redirect_stdout(pipe) do
    write(stdout, String(rand(['a':'z'; '\n'], 100_000)))
    close(Base.pipe_writer(pipe))
end

result = read(pipe, String)
wait(writer) # Not really necessary but cleaner

```

It’s maybe a bit hacky to use the unexported `pipe_writer` function, and to close the stream before the end of `redirect_stdout`… This should all be made simpler, see the following issues:

> <https://github.com/JuliaLang/julia/issues/12711>
>
> It'd be really nice to be able to redirect stdout/err to an IOBuffer:
> 
> \`\`\`
> julia…\> b = IOBuffer()
> IOBuffer(data=UInt8\[...\], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)
> 
> julia\> redirect\_stdout(b)
> ERROR: MethodError: \`redirect\_stdout\` has no method matching redirect\_stdout(::Base.AbstractIOBuffer{Array{UInt8,1}})
> \`\`\`
> 
> I thought I saw another similar issue the other day, but was unable to locate it.
> 
> (Aside/another issue it would be nice to write this as a do block.)

> <https://github.com/JuliaLang/julia/issues/28679>
>
> The usability of the \`redirect\_stdout\` etc. functions got a big boost with the \`…do\` versions, but there are still some gotchas that make them often inconvenient, and I think they can be improved without any breaking changes:
> 
> \### Sometimes you just want to throw away output
> 
> This could be written as:
> \`\`\`julia
> redirect\_stdout(devnull) do
> # noisy code here
> end
> \`\`\`
> 
> This was proposed in #22879, and seems like it would need a cross-platform way to open a null file. It \[looks like\](https://stackoverflow.com/questions/313111/is-there-a-dev-null-on-windows) on windows you can \`open("nul", "w")\`. So this could be implemented as something like:
> 
> \`\`\`julia
> function Base.redirect\_stdout(dofunc::Function, ::Base.DevNullStream)
> nullfile = @static Sys.iswindows() ? "nul" : "/dev/null"
> open(nullfile, "w") do io
> redirect\_stdout(dofunc, io)
> end
> end
> \`\`\`
> 
> \### It's come up before that the \`redirect\_\` functions don't support more generic \`IO\` types
> 
> This was mentioned in the above issue, and also in #12711. It would look like:
> 
> \`\`\`julia
> io = IOBuffer()
> redirect\_stdout(io) do
> # code
> end
> seekstart(io)
> read(io) # get the info
> \`\`\`
> 
> \### Capturing to a string is a multi-step process
> 
> It's common in testing to check that your code is outputting what you expect, so an easy way to capture to a string would be useful. This could be:
> 
> \`\`\`julia
> output = redirect\_stdout() do
> # test code here
> end
> \`\`\`
> 
> This implementation for both these last two could look very similar to the \[\`@capture\_out\` macro\](https://github.com/JuliaIO/Suppressor.jl/blob/38deabab1567be98ac6a2a39744ec582f9047e3e/src/Suppressor.jl#L129-L152) from Suppressor.jl except that there's no need for it to be a macro (originally those were macros so that the macro didn't affect the expression's scope, but that got blown out of the water when we added the \`try...finally\` block).:
> 
> \`\`\`julia
> function Base.redirect\_stdout(dofunc::Function, io::Union{IO,Nothing}=nothing)
> if ccall(:jl\_generating\_output, Cint, ()) == 0
> original\_stdout = stdout
> out\_rd, out\_wr = redirect\_stdout()
> out\_reader = @async if io === nothing
> read(out\_rd, String)
> else
> read(out\_rd)
> end
> end
> 
> out = io === nothing ? "" : UInt8\[\]
> 
> try
> dofunc()
> finally
> if ccall(:jl\_generating\_output, Cint, ()) == 0
> redirect\_stdout(original\_stdout)
> close(out\_wr)
> out = fetch(out\_reader)
> end
> end
> 
> if io === nothing
> out
> else
> write(io, out)
> nothing
> end
> end
> \`\`\`
> 
> TBH I'm not 100% sure what the \`if ccall(:jl\_generating\_output, Cint, ()) == 0\` checks are for, I think they came from \[some of @keno's code\](https://github.com/JuliaLang/JuliaParser.jl/blame/985c2b5e23af26b9a4f1c66b3da5ea96c90f4eb2/bin/repl.jl#L7-L10)
> 
> These methods are all currently errors, so it seems like they could be candidates for a 1.x release if they seem like OK ideas.

---

<div class="post-metadata">

**Author:** ![Euhan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/euhan/32/36548_2.png) [@Euhan](https://discourse.julialang.org/u/Euhan)\
**Post date:** [December 20, 2022, 6:00pm UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/7 "2022-12-20T18:00:40Z")

</div>

> [@sijo](#):
>
> ```julia
> pipe = Pipe()
> writer = @async redirect_stdout(pipe) do
> write(stdout, String(rand(['a':'z'; '\n'], 100_000)))
> close(Base.pipe_writer(pipe))
> end
> 
> result = read(pipe, String)
> wait(writer)
> 
> ```

In 1.8.3 it gives:  
ArgumentError: Base.PipeEndpoint(RawFD(4294967295) init, 0 bytes waiting) is not initialized

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 21, 2022, 9:37am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/8 "2022-12-21T09:37:51Z")

</div>

Oops that’s because this code has a race condition. The `pipe` object must be initialized by `redirect_stdout` before it’s used in the `read(pipe, String)` call. But the order of execution is undefined here: Julia can decide to start executing the async task before `read` (1.7 behavior apparently), or it can start executing `read` before the async task (1.8 behavior).

To fix the race condition we can add some explicit synchronization:

```julia
pipe = Pipe()
started = Base.Event()
writer = @async redirect_stdout(pipe) do
    notify(started)
    write(stdout, String(rand(['a':'z'; '\n'], 100_000)))
    close(Base.pipe_writer(pipe))
end

wait(started)
result = read(pipe, String)
wait(writer) 
println(result)

```

(By the way, the first example in that same message had an error: it should be `pipeline(pipe, `wc -l`, output)` instead of `pipeline(pipe, `wc -l`, result)`, but I can’t edit it anymore.)

---

<div class="post-metadata">

**Author:** ![ess3sq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ess3sq/32/26834_2.png) [@ess3sq](https://discourse.julialang.org/u/ess3sq)\
**Post date:** [December 21, 2022, 9:50am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/9 "2022-12-21T09:50:58Z")

</div>

Is this not equivalent to not using an asynchronous task at all?

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 21, 2022, 10:00am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/10 "2022-12-21T10:00:33Z")

</div>

No: without an asynchronous task, the code in `redirect_stdout` has to finish before we start reading. I expect that the `write` will block because the pipe will be full before all data is written, so the whole program will get stuck.

With the asynchronous task, the `read` call can start consuming the pipe as the `write` call is still running and feeding more data to the pipe.

In other words: the async task with synchronization makes sure we don’t read before the producer starts. Having no async task at all would delay reading until the producer finishes (which will never happen if the data doesn’t fit in the pipe buffer).

---

<div class="post-metadata">

**Author:** ![ess3sq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ess3sq/32/26834_2.png) [@ess3sq](https://discourse.julialang.org/u/ess3sq)\
**Post date:** [December 21, 2022, 10:03am UTC](https://discourse.julialang.org/t/a-minimal-example-with-base-redirect-stdout/64245/11 "2022-12-21T10:03:24Z")

</div>

I see, thank you