Strange characters in windows terminal

General Usage
1

I have the following code:

function git_pull()
    result = 0
    try
        msg = readlines(`$git pull --rebase`)
        println(msg)
    catch e
        result = 1
    end
    result
end

If I call this function from the repl in vscode I get the following output:

julia> git_pull()
["Already up to date."]
←[0m0

This happens only on one of two Windows 10 computers I have.
Output on Linux

julia> git_pull()   
["Current branch main is up to date."]
0

Where are these strange characters coming from?

2

Found a solution for Python, but not yet for Julia: python - How to use the new support for ANSI escape sequences in the Windows 10 console? - Stack Overflow

1 Like
3

Perhaps a filter function that removes all ANSI escape sequences from a string would already help… But how could I implement it?

4

StringManipulations.jl has a remove_decorations function that can remove ANSI escape sequences from a string.

julia> str = "Test 😅 \e[38;5;231;48;5;243mTest 😅 \e[38;5;201;48;5;243mTest\e[0m";

julia> remove_decorations(str)
"Test 😅 Test 😅 Test"

ANSIColoredPrinters.jl also has a PlainTextPrinter that removes ANSI escape codes, though slightly less convenient to use.
StringManipulations accepts a normal string and returns a string, so it just requires println(msg) to become println(remove_decorations(msg)).

3 Likes
5

I tried:

function git_pull(throw=false)
    result = 0
    if throw
        msg = readlines(`git pull --rebase`)
        for line in msg
            println(remove_decorations(line))
        end
    else
        try
            msg = readlines(`git pull --rebase`)
            for line in msg
                println(remove_decorations(line))
            end
        catch e
            result = 1
        end
    end
    result
end

But the result is the same:

julia> git_pull()
["Already up to date."]
←[0m0

Perhaps this is not an ANSI escape sequence?

What else can I try?

The problem is that this brakes the terminal menu that I am using…

6

Can you reproduce this outside of VS Code?

7

Could you print out the bytes of the result, like

line |> codeunits |> println

and

remove_decorations(line) |> codeunits |> println

and show the result? I’m fairly certain by now that the escape sequence is not actually part of the string, but is an artifact of printing. This would help make sure that’s true.

Do you mean that the output in the other Windows computer, in VS Code REPL, doesn’t print these extra characters? If so, could you compare the startup.jl files in these two computers? (eg. redefining println to use printstyled could cause this.)

8

Can’t you use the SetConsoleMode solution from that answer? Something like this in your startup.jl:

if isinteractive()
    let consoleout = ccall(:GetStdHandle, stdcall, Ptr{Cvoid}, (Int32,), -11)
        ccall(:SetConsoleMode, stdcall, Cint, (Ptr{Cvoid}, Int32), consoleout, 0x0001 | 0x0004 | 0x0008)
    end
end
9

I will investigate further on Monday, don’t have access to the laptop that shows this issue right now…

10

Yes, we just tried it using the “windows command prompt”, and the output is the same. So this is not a vscode related issue.

11

Well, we are not using any startup.jl files.

I tried:

hex(n) = string(n, base=16, pad=3)
function git_pull2()
    result = 0
    try
        msg = readlines(`git pull --ff`)
        for line in msg
            clean_line = remove_decorations(line)
            for char in clean_line
                print(hex(Int64(char)), " ")
            end
            println()
        end
    catch e
        result = 1
    end    
    result
end

Output:

git_pull2()
041 06c 072 065 061 064 079 020 075 070 020 074 06f 020 064 061 074 065 02e
←[0m0

So the strange characters are not part of the string that I am printing…
(Still need to check if any control character is in this string…)

But where else do the come from?

12

git prints the odd characters to stderr.

13

Ah, that’s pretty odd, to print the reset sequence even if it hasn’t actually printed anything else to stderr. So would readlines(pipeline(`git pull --ff`, stderr = devnull)) get rid of the issue?

14

Would there be the risk that I don’t see error messages any longer if I do that?

15

Yeah, any error messages from git would be siphoned off into the void. You can instead change devnull to be a file instead, or if you just want to print it, something like

  io = IOBuffer()
  readlines(pipeline(`git pull --ff`, stderr = io)) 
  git_err = remove_decorations(String(take!(io)))
  println(git_err)

should work.

1 Like
16

All working fine now. Thanks everybody for the good support! :slight_smile:

Final code:

function exec(command)
    io = IOBuffer()
    msg = readlines(pipeline(command, stderr = io)) 
    err = remove_decorations(String(take!(io)))
    msg, err
end

function git_pull(throw=false)
    function pull()
        result = 0
        msg, git_err = exec(`git pull --ff`) 
        for line in msg
            println(remove_decorations(line))
        end
        if length(git_err) > 0
            result = 1
            println(git_err)
        end
        result
    end

    result = 0
    if throw
        result = pull()
    else
        try
            result = pull()
        catch e
            result = 1
        end
    end
    result
end
1 Like