Help question with running Stan.jl in VSCode on Windows

General Usage
1

Hi,

When I run below script in VSCode on Windows:

using CmdStan, Statistics
bm = "
  data { 
    int<lower=1> N; 
    int<lower=0,upper=1> y[N];
  } 
  parameters {
    real<lower=0,upper=1> theta;
  } 
  model {
    theta ~ beta(1,1);
    y ~ bernoulli(theta);
  }
";
data = Dict("N" => 10, "y" => [0, 1, 0, 1, 0, 0, 0, 0, 0, 1])
sm = Stanmodel(name="bernoulli", model=bm, printsummary=false)
sm |> display
rc, samples, cnames = stan(sm, data; summary=false, CmdStanDir=CMDSTAN_HOME);
nt = CmdStan.convert_a3d(samples, cnames, Val(:namedtuple))
mean(nt.theta, dims=1) |> display

then, after the call to stan(), screen make-up characters are showing up, e.g. in https://user-images.githubusercontent.com/29783567/111507734-f8ea4680-8718-11eb-9a43-33a344851bfb.png .

A typical sequence would be: ā€œ<-[?2004l<-0m<-0m(tree depth__ = [1.0 ā€¦ā€ where the last part is correct and is the beginning of the nt NamedTuple.

I suggested to switch to Powershell (which has helped in the past, but in this case it doesnā€™t seem to).

Is this a VSCode issue? Is there another development setup with a separate Powershell that maybe works?

I donā€™t have access to a Windows machine nor do I normally work in VSCode but if I bring it up on macOS it works fine.

Thanks for your help!

2

I think what is happening there is that some Julia code is spawning some command line executable that is changing the console mode of the (shared) console, and in particular is switching off support for escape sequence support in the console. But Julia is still sending control sequences to the console, and then you get that kind of corrupted output.

I think the proper fix for this would probably be that Julia stops spawning child process in a way that gives them full access to the main console, because at the end of the day any of these child processes can always change something about the console that will break output. Instead, Julia probably should spawn child processes connected to a pseudo console instead, and then make sure output that the child process displays on the pseudo console is displayed in the main console. That way Julia could probably guarantee that a child process canā€™t cause this kind of problem.

Not sure whether you have control over the binary that is spawned, but you could also try to prevent it from changing the console modeā€¦

2 Likes
3

Thank you David! Very insightful, and hints on ways I could try to fix this, e.g. direct all output from the spawned ā€˜cmdstanā€™ executable to stderr or so as I donā€™t need it.

4

Yeah, that might work, but not sureā€¦ Just a bit more detail: I think the child process is almost certainly calling SetConsoleMode (SetConsoleMode function - Windows Console | Microsoft Docs) without the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag set, and that switches the entire console from processing control sequences to just displaying them as if they were text inputā€¦

2 Likes
5

I should be able to find that in the sources and Iā€™ll check back with the Stan team. Thanks again.

1 Like
6

About a week ago I helped out someone else and he confirmed after some back and forth trying:

ā€œI now confirmed the walkthroughs for both CmdStan and StanSamples work if I start julia from Command Prompt or from Powershell, but fail if I start julia from git bash.ā€

I wonder if somehow inside VSCode we are still starting Julia REPL in git bash as in this case it doesnā€™t seem to work.

Did a quick check in the cmdstan and stansummary C++ sources but no visible call to SetConsoleMode so need to dig deeper into that possibility.

7

The integrated ā€œJulia REPLā€ doesnā€™t go through a shell at all, we just start Julia directly.

8

Thank you.

ENV on my test setup for VSCode shows:

julia> ENV
Base.EnvDict with 52 entries:
  "USER"                                => "rob"
  "__CFBundleIdentifier"                => "com.microsoft.VSCode"
  "COMMAND_MODE"                        => "unix2003"
  "DISPLAY"                             => "/private/tmp/com.apple.launchd.dxa1j0rGUj/org.macosforge.xquartz:0"
  "LOGNAME"                             => "rob"
  "PATH"                                => "/Users/rob/.julia/packages/Homebrew/l8kUw/deps/usr/opt/gettext/bin:/Users/rob/Library/Python/3.6/bin:/usr/local/opt/bison/bin:/Applications/MacVim.app/Contents/binā€¦
  "SSH_AUTH_SOCK"                       => "/private/tmp/com.apple.launchd.X5Sa1Alq4C/Listeners"
  "SHELL"                               => "/bin/zsh"
  "HOME"                                => "/Users/rob"
  "__CF_USER_TEXT_ENCODING"             => "0x1F5:0:0"
  "TMPDIR"                              => "/var/folders/l7/pr04h0650q5dvqttnvs8s2c00000gn/T/"
  "XPC_SERVICE_NAME"                    => "application.com.microsoft.VSCode.78777835.78777841"
  "JULIA_PKG_SERVER"                    => "https://us-east.pkg.julialang.org"
  "JULIA_PKG3_PRECOMPILE"               => "true"
  "JULIA_MPI_PATH"                      => "/usr/local/Cellar/open-mpi/4.0.2"
  "JULIA_SPECIALFUNCTIONS_BUILD_SOURCE" => "true"
  "JULIA_SVG_BROWSER"                   => "Google Chrome.app"
  "JAGS_HOME"                           => "/usr/local/bin/jags"

I asked Justin what ENV looked like on his setup and it shows:

...
  "JULIA_EDITOR"                    => "\"C:\\Users\\Justin\\AppData\\Lā€¦  "JULIA_NUM_THREADS"               => ""
  "TERM_PROGRAM"                    => "vscode"
  "TERM_PROGRAM_VERSION"            => "1.54.3"
  "LANG"                            => "en_US.UTF-8"
  "COLORTERM"                       => "truecolor"
...

and, as you point out, no shell. But, unix thinking probably, VSCode on Windows must use some kind of environment to spawn the Julia REPL which works fine until I call stan() and immediately after that call the terminal doesnā€™t handle the the escape characters anymore.

As mentioned above, I checked the cmdstan and stansummary sources, to no avail, my next step is to see whatā€™s included in the to C++ compiled/translated Stan Language program.

9

I believe this may be the same or similar to this issue reported by me:

https://discourse.julialang.org/t/reset-repl-terminal-mode-to-working-default-settings/51892

I wanted to use the REPL shell mode to call cygwin programs:

shell> ls
20201105_12  20201106_11  Manifest.toml  Project.toml  io_caen  movers  test_proj  tutorial

ā†[7Cā†[32mā†[1mjulia> ā†[0mā†[0m

but the teminal mode in the julia repl gets screwed up. A sort of work around is to use the windows cmd.exe to indirectly call the desired program like this:

shell> cmd /c ls
20201105_12  20201106_11  Manifest.toml  Project.toml  io_caen  movers  test_proj  tutorial

julia>

Note that the line mode at the Julia REPL is now correct. After seeing your issue, I think maybe the problem is that the julia mode in the REPL is letting terminal settings from shell (and presumably help) mode leak in. Would adding a reset to the julia the current julia terminal mode fix the problem?

10

Thanks a lot! I should be able to do that as well!

11

Well, not quite a solution, but apparently Stan.jl (based on StanSample.jl, StanOptimize.jl, etc.) works fine in VSCode on Windows.

I need to check where/why CmdStan.jl is not handling above problem correctly. The cmd /c part had been added some time ago but maybe there is a path where this is not done right.

CmdStan.jl is in maintenance mode, so itā€™s not a top priority on my list of things to fix in the combination CmdStan + VSCode + Julia REPL given that Stan.jl is working in that setting.

As always, from all issues you take something away. In this case I took another look at VSCode and, compared to a year ago when I wanted to step away from TextMate and in the end opted for Sublime Text, this time around liked it quite a bit.

Thanks for the great help on this issue!

1 Like