Vscode 'send to active terminal' for persistence and access to scheduler-controlled HPC nodes?

Tooling VS Code
, , ,
1

Question:

How do we make @command:language-julia.executeJuliaCodeInREPL control the active terminal instead of the REPL? This would be analogous to setting "r.alwaysUseActiveTerminal": true for Yuki Ueda’s R extension below.

Background:

When using julia on a cluster (via the vscode remote-ssh extension), the following cases are better served by a REPL started in the active terminal, rather than the julia-vscode REPL (language-julia.startREPL):

  1. interactively running something that is non-trivial (on a compute node via slurm srun), where the actual compute node cannot be remotely-accessed (WAN access to HPC compute nodes is always restricted for accounting and networking reasons)
  2. REPL-persistence is required (eg via tmux/screen) when a disconnect may interrupt the session (laptop battery dead, etc)

To address this need, some vscode extensions allow the user to decide whether to send code from the editor to the active terminal (eg when it is remote, multiplexed and therefore persistent) or to the REPL (eg when working locally).

1 Like
2

We’ve solved that problem differently. You can connect to an arbitrary REPL process with Julia: Connect External REPL; afterwards, all code will be evaluated there.

For your specific use case, check out the julia.persistentSession.enabled setting, which allows the extension to manage its own tmux session for your Julia REPL without any interaction on your part (which may or may not work in your case).

3

Thanks providing this awesome extension, @pfitzseb!

Please see “2. Compute node…” below.

1. Login node (Connect External REPL succeeds):

The External REPL is running on a login node that is accessed via ssh.
This host will automatically terminate connections that are running cpu or memory-intensive tasks (stuff requiring Distributed::addprocs(), etc).

Steps to reproduce:

  1. start vscode and connect to remote host eg ssh me@hpc3.rcic.uci.edu (a line in my ~/.ssh/config)
  2. within the vscode terminal, call julia to open an external REPL
  3. within command palette, call Julia: Connect External REPL
  4. paste the copied line to the REPL (3 above)
[me@login-i16:/dfs6/pub/me/code/example_project] $julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> pushfirst!(LOAD_PATH, raw"/data/homezvol2/me/.vscode-server/extensions/julialang.language-julia-1.2.8/scripts/packages");using VSCodeServer;popfirst!(LOAD_PATH);VSCodeServer.serve(raw"/tmp/vsc-jl-repl-920b1264-ff19-4996-b7ba-9ec2323b29c6"; is_dev = "DEBUG_MODE=true" in Base.ARGS, crashreporting_pipename = raw"/tmp/vsc-jl-cr-5bbee6d9-b610-432f-b8c6-d4cf5b6712bd");nothing # re-establishing connection with VSCode

julia>

2. Compute node (Connect External REPL fails):

This time, the External REPL is running on a firewalled compute node.
As usual on compute clusters, you can’t access compute nodes via ssh (LAN or WAN).
Instead I must call srun. The scheduler then selects a host and drops me into a local terminal.

Steps to reproduce:

  1. start vscode and connect to remote host eg ssh me@hpc3.rcic.uci.edu (a line in my ~/.ssh/config)
  2. within the vscode terminal, call srun --account ... to get a terminal on a compute node
  3. within the vscode terminal, call julia to open an external REPL
  4. within command palette, call Julia: Connect External REPL
  5. paste the copied line to the REPL (4 above)
[me@login-i16:/dfs6/pub/me/code/example_project] $srun --account=myaccount --partition=free --cpus-per-task=40 --mem-per-cpu=7800M --pty bash -i
[me@hpc3-17-00:/dfs6/pub/me/code/example_project] $julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> pushfirst!(LOAD_PATH, raw"/data/homezvol2/me/.vscode-server/extensions/julialang.language-julia-1.2.8/scripts/packages");using VSCodeServer;popfirst!(LOAD_PATH);VSCodeServer.serve(raw"/tmp/vsc-jl-repl-21812c16-2650-44c5-be4d-779ca5392e72"; is_dev = "DEBUG_MODE=true" in Base.ARGS, crashreporting_pipename = raw"/tmp/vsc-jl-cr-5bbee6d9-b610-432f-b8c6-d4cf5b6712bd");nothing # re-establishing connection with VSCode
ERROR: IOError: connect: no such file or directory (ENOENT)
Stacktrace:
 [1] wait_connected(x::Base.PipeEndpoint)
   @ Sockets /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Sockets/src/Sockets.jl:532
 [2] connect
   @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Sockets/src/Sockets.jl:567 [inlined]
 [3] connect
   @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Sockets/src/PipeServer.jl:97 [inlined]
 [4] serve(args::String; is_dev::Bool, crashreporting_pipename::String)
   @ VSCodeServer ~/.vscode-server/extensions/julialang.language-julia-1.2.8/scripts/packages/VSCodeServer/src/VSCodeServer.jl:89
 [5] top-level scope
   @ REPL[1]:1
2 Likes
4

Turns this functionality I was asking for is already natively supported in vscode, as noted by @tshort in a related julia-vscode feature request.

So all that is needed is to retrieve the json id from the palette command above, and add it to ~/.config/Code/User/keybindings.json:

     {
        "key": "ctrl+enter",
        "command": "workbench.action.terminal.runSelectedText",
        "when": "editorTextFocus && editorLangId == 'juliamarkdown'"
    },
    {
        "key": "ctrl+enter",
        "command": "workbench.action.terminal.runSelectedText",
        "when": "editorTextFocus && editorLangId == 'julia'"
    }

I’ve just confirmed this works within a slurm interactive session on a compute node as in (2) above.

1 Like
5

Hi!
Thank you for posting this solution :slight_smile: I wondered if there is a way to make it more adapted to the Julia syntax. For example, doing ctrl+enter with that config in the first line of a function definition will only send that line to the terminal rather than sending the whole function. Is there a way to fix that?
Best regards,