How to attach to an existing remote REPL?

Hi,
In my workflow I need to run several simulations which make use of the DifferentialEquations package.
To automate running the simulations I need to start the simulations from command line (and not within the Julia REPL):
julia sim1.jl
julia sim2.jl
julia sim3.jl
etc…
The problem I’m facing is that the startup time is large and it constitutes the major part of running “julia simX.jl”. Using PackageCompiler to shorten startup time is not an option because I cannot make it work reliably on all the workstations I have to use to run the tests.
After some thinking I started wondering if it is possible to connect to an existing REPL and run the tests sequentially from my remote connection. I looked at the documentation of the Distributed package and it seems like there is all the stuff I need but I couldn’t figure out how to get what I wanted. What I’d like to achieve is the following (example for 2 simulations)

  1. connect to the REPL
  2. run the simulation 1
  3. disconnect from the REPL. The REPL continues to exist.
  4. connect to the same REPL
  5. run the simulation 2
  6. disconnect from the REPL.

Is it possible to attach the current julia process to an existing REPL (remote or on the same computer)?
If it is not possible, what can I do to speed up my work?

Thank you
Carlo

Carlo, this is NOT a direct answer to your question. An expert will come along in a minute.

If you use an ssh connection to a remote system you can use a utility called ‘screen’
A screen session will keep an application running and you can disconnect and reconnect to the terminal.

ALso - if you have a REPL session running you can avoid stopping it and restarting.
Look at the Revise package by Tim Holy

Have you considered using a jupyter notebook?
That should allow using a remote julia session (which stays alive)
Or is this not an option for you?
You can check this out on juliabox
Apologies if I misunderstood your problem.

screen or tmux are the canonical solutions to this problem.
Juno can automate that process for you pretty well, if you want to give it a go.

2 Likes

Thank you for the suggestion but I really need to start my simulations from the command line. Carlo

Actually I was planning to use Revise if I manage to get this working. Thank you
Carlo

@johnh and @pfitzseb
I knew about screen but it didn’t came into my mind to solve the problem. I think with screen I could get what I want but if I could attach and detach to a REPL using the same tools used for distributed computing I could also get the results of my simulations which could help me understand if everything went ok or not. Probably I can do it also with screen but it would be more complicated.

Attaching a REPL to a running remote process is in principle possible but no one has done the work to make it happen. Feel free to open a feature request issue on GitHub. Note that there are pretty hefty security implications of allowing such a thing, so it’s not just a matter of making it work, but also of making it work safely.

Thank you for the feedback. Would it be a quite unsafe thing also using the Distributed.jl package infrastructure? I don’t see a big difference between attaching to a running Julia process and distributed computing with Julia.

True, being able to start a REPL on any worker would be safe and probably easier to implement since the “RPC” already just works.