Hi all, I recently released RemoteREPL.jl which makes it simple to connect your local Julia REPL to a remote Julia process. I’ve mentioned the package in a couple of places but thought it could do with an announcement thread.
RemoteREPL can attach a REPL to any cooperating Julia process which has called serve_repl(). The client REPL started with connect_repl() supports executing code in the Main module of the remote process, has tab completion, help mode with the ? prefix, and automatically reconnects if the connection is dropped.
For remote machines, RemoteREPL automatically tunnels traffic over SSH so should have decent network security. (For multi-user environments there’s still some security caveats.)
Demo:
Thanks to @Mason for ReplMaker.jl which makes setting up new repl modes quite easy.
this is the coolest thing I’ve seen in this type of tooling for a while. Now I can run Julia session on cluster nodes (that cannot be ssh-ed into) and debug my script this way! Do you think it’s possible to support socket-file protocol in addition to ssh? (see ClusterManagers.jl for example)
All communication currently happens over a normal TCP socket (SSH is only used for tunneling the connection over an insecure network). It should be fairly easy to set up a connection via a socket file rather than a TCP socket, it’s just a matter of doing the plumbing work. Without knowing more about your cluster I’m not quite sure how this would help you though. Which cluster manager are you using from ClusterManagers?
Slightly related - RemoteREPL also supports alternative tunnels (a) for Kubernetes via kubectl and (b) for Amazon EC2 session manager thanks to @Nishanth_Kottary.
sorry for not being clearer. The login node (I can ssh into) and the cluster note share some part of the storage (other wise cluster can’t read config files or data I prepared). Even thought I cannot directly ssh into a cluster node easily.
Right, so there’s a shared filesystem mounted on the login and the compute nodes. This won’t help with socket communication though because socket files can’t be used to communicate between different machines. (If I understand correctly, they’re not like normal files at all, mainly they only exist as a path name to identify a kernel resource on a single machine.)
If the compute nodes are on the same network as the login node you could use the RemoteREPL protocol directly rather than an ssh tunnel. Unfortunately this is completely insecure in a shared cluster environment and I don’t recommend it. We could add a shared token (“password”) to the connection header so that it’s as secure as Julia’s existing Distributed protocol (which is to say, not very secure IIUC). Then it could be justified to use RemoteREPL anywhere that Distributed.jl itself is currently used.
What you really need for your use case is to connect directly to the RemoteREPL port. It’s technically possible to expose this port, but in the current version there’s no security: it allows any user on the network to log into your node and execute arbitrary code.
Ideally we’d add security to the RemoteREPL protocol itself. At least with a basic token, or better with libssh2_jll — in that case, it would be like RemoteREPL acting as its own ssh server, though I’m not sure how much work that would be.
