Switching from Jupyter to a proper IDE

I think the plot pane in the julia extension should just work with the SSH extension. We don’t have inline plots in general (local or remote).

I can confirm the plot plane is working over the SSH extension as @davidanthoff said. I use it a lot!

Thanks for suggestions, I looked in more details into VS Code Remote-SSH and other extensions like hydrogen for Atom. Even though they are more convenient for remote development than just Juno remote, but it looks like none of those solutions a) persist plots and other outputs, so they must be recomputed each time and hope the result doesn’t change; and b) allow “seamless switching” between two computers, i.e. having the very same set of files/sessions active when you move from one PC to another and connect to the same server.
So it’s nice to see IDEs getting features previously unique to IPython/Jupyter, but it seems that at least for my usage scenario (see 1st post for details) IDEs are not there yet.

You could use VS code remote (or some other editor with this feature) to develop and make changes to your package/module/jl file, but still do the analysis and plotting using Jupyter (+ Revise.jl ). This way you get the best of both worlds.

EDIT: Also, you could consider using emacs/vim inside a tmux session running on the remote machine in order to have a persistent editor session when you switch machines.

1 Like

emacs/vim inside a tmux session running on the remote machine

I can imagine emacs being able to keep persisntent plots etc next to corresponding code blocks (however didn’t use those editors myself), but tying everything to a remote session means that without reasonably fast and reliable internet connection it’s impossible to view existing code/results/… In Jupyter obviously one can examine and edit the already loaded page. I often experience poor and dropping connection eg while in transport, and jupyter deals with this pretty well. Also typing delays become very uncomfortable when using remote session like this on mobile connection.

While it is indeed possible to get plots and Jupyter like experience in emacs (for example using emacs-jupyter + org-mode, not sure if plots can work inside a tmux session though), here I was suggesting it only as replacement to the VS code part of the workflow I suggested and not for the Jupyter part.
But yes, I guess that with this setup you would not be able to keep working in the editor if your connection dropped (I’ve heard that using mosh instead of ssh is supposed to somehow help with poor connections but not sure exactly what it provides).

I am using Atom/Juno with “ftp remote edit” plugin, and Julia running in a tmux session on the host workstation. So when I connect to it from different editors, the Julia process stays the same, with its workspace. This works reasonably well for me, and I think covers all the use cases that you need. Specifically for plots, you could prepare them in advance, and then just call display(plt) when you reconnect from a new editor. Plot pane in Atom works over remote connection.

I thought of a similar setup, unfortunately there are several issues with it: a) as I understand, you cannot easily have several independent julia sessions, each automatically corresponding to separate “workspaces”, e.g. directories with source files; b) more importantly, dealing with plots this way doesn’t work across julia restarts, requires to keep references to all the plots, giving them names and not forgetting which is which, and overall is just way too “manual” - compare to notebooks, where all the output (including plots) is unambiguously tied to the code producing it and stored together “forever”.

Atom also has a Hydrogen plugin, that apparently allows you to connect to remote Jupyter kernels. I have not tested it in anger, though.

It sounds as if you really want/need to work with an interactive computational notebook, so why not just continuing working with a Jupyter notebook :slight_smile: ?
Again, I personally think that the hybrid workflow is the best here. Do things which are best done in notebooks in Jupyter (plotting, data analysis) and other things (development of packages) in your editor of choice.

3 Likes

Yes, I really do want and need, but the reasons I’m trying to find something else are also outlined in the first post: jupyter really lacks in text editing capabilities - even auto indent often works incorrectly, and in debugging/code navigation.

Sounds to me like what you want is just to do your work via ssh on a remote server, possibly in a docker container. It doesn’t really sound to me like anything you’re doing is unique to Jupyter (full disclosure: I despise Jupyter).

  • Sounds like you want to run Julia in a docker container, you can do this anywhere, it has nothing to do with Jupyter.
  • For vim and emacs at least, it is trivial to reproduce a setup, see for example my nvim setup file. I’d imagine that any IDE’s that are any good have a similarly easy to reproduce setup.
  • This does seem like one area in which JupyterLab may have a slight advantage. You don’t typically need a ton of bandwidth for ssh but you do need some and if you have a bad connection lag can be frustrating.
  • REPL output doesn’t disappear and even stores a history between sessions in .julia/logs/repl_history.jl.
  • Nothing is stopping you from running arbitrarily many instances of Julia.
1 Like

Maybe we could get https://github.com/krassowski/jupyterlab-lsp to work with https://github.com/julia-vscode/LanguageServer.jl? Maybe that would make JupyterLab more usable?

For me personally the fact that Jupyter lab capbilities are limited is actually a feature- it incentivizes me to refactor my code into more organized packages so I can work on it with a normal text editor, instead of keeping everything in huge messy notebooks :slight_smile: .

This jupyterlab-lsp looks cool in any case. For jumping to definition there is also this simpler extension (from the same author of the lsp one) which seems to just work.

For what it’s worth, emacs-jupyter has full support for remote kernels–no tmux session necessary. What the op is requesting sounds like it would be perfectly served by emacs-jupyter + org-mode + tramp, but I’m hesitant to suggest jumping ship to emacs for this kind of setup because there are so many interconnected moving parts that getting it working reliably might be a bit fiddly for somebody new to the editor.

But @aplavin, for what it’s worth, emacs-jupyter + org-mode + tramp satisfies every single point in your original post.

2 Likes

Actually, before jupyter I used a similar workflow (for python, but it doesn’t matter here). And when jupyter (especially jupyterlab) appeared, it was like fresh air after that.

I think there is some misunderstanding here - by “same layout between different computers” I meant the same files open in the same way, not just having the same editor settings.

This file contains inputs only, no outputs (at least for me). And of course there are no plots.

Sure, but you always need to manually choose which instance to use. While in jupyter there is a clear correspondence between notebook and backend instance, so they cannot be mixed up.

Indeed this looks pretty close to what I imagined, closer than any other solutions I saw. Maybe at some point I will seriously try emacs :slight_smile:

It sounds to me like the core issue is that with JupyterLab you are doing everything from a single server, and when you are using IDE’s you are doing it from multiple different servers. You can have the same reproducability with most IDE’s if you just did that from a single server as well.

Granted, if you’re connectivity is not good enough for an ssh session, that’s definitely an issue I don’t know how to overcome. If this really is a persistent issue for you you may have no choice but to stick with JupyterLab.

Is there a simple way to reproduce the exact window layout and list of open buffers between different machines? This seems to be also something OP wishes for.
I’m aware of save-desktop mode (or something like this) to reproduce the state when restarting emacs on the same machine, could one somehow share this between machines?

I haven’t tried it my self, but I think saving a perspective file on a remote machine should “just work”. There would probably be an issue if you access files by different paths on different machines, though (/ssh:user@myurl#20:/path/to/file.org vs smb:user%domain@myurli:/path/to/file.org or something like that).

EDIT: just tested this setup quickly, and it seems to work as expected.