Hi! I’m using JupyterLab for the majority of my work in Julia, and it is definitely lacking in capabilities of IDEs like Juno and Julia plugin for VSCode - like debugging, variable inspection, and so on. Even autoindent doesn’t always work correctly in jupyter. But it turns out not so easy to switch from jupyter, at least I couldn’t find a reasonably documented workflow for my usecase. Main advantages of jupyter as I see it are basically the following:
The whole julia (and python - mostly used by pycall) setup is run in docker (for easy updates and reproducibility) on one fast machine, where all the data resides as well. Then from any place (2-3 other computers) I can easily use it without worrying about any kind of synchronization. These “other” computers don’t have all the libraries and packages installed, and don’t store my code - it’s located all in one place.
The very same layout between different computers - e.g. come home, open jupyter, and see exactly the same as you did at work.
No matter if the internet is fast or slow (think 3G-scale ping) it doesn’t affect typing and looking at previously-generated results and plots. Only the new outputs are slowed down (obviously). And even without internet connection I can see all previous outputs.
Permanent (until manually removed) display of code blocks and their corresponding results together.
Multiple independent julia sessions simultaneously, each with its own set of code blocks and results, easily switchable back and forth.
Looking at Atom and VSCode docs I couldn’t easily find how to setup something similar there. Do you think it is possible now, or going to be possible when some planned developments to those IDEs are implemented?
To mirror your general architecture you could use Juno connected to a remote server (see this discussion).
The advantage would be that all the libraries and data for your computations would reside in the server, although the script would still be in your local machine, but at that time you could use on your server NextCloud or something similar in order to have the script automatically synchronised on the server as well.
This can make sure files (scripts) are synced, but it’s just one point from the list above… And it’s not clear from the docs if remote server connection in Juno supports plots/interaction?
Take a look at the Visual Studio Online and Remote - SSH extension for VS Code. They make for an incredibly smooth remote working experience where all your code, setup, data lives on one machine but you edit it from a different one. It should also all work with the Julia extension for VS Code.
This does look very interesting, somehow I missed those extensions. Definitely going to try! However, it looks like at least Remote - SSH doesn’t support inline or side-by-side plots. Is that true?
Thanks for suggestions, I looked in more details into VS Code Remote-SSH and other extensions like hydrogen for Atom. Even though they are more convenient for remote development than just Juno remote, but it looks like none of those solutions a) persist plots and other outputs, so they must be recomputed each time and hope the result doesn’t change; and b) allow “seamless switching” between two computers, i.e. having the very same set of files/sessions active when you move from one PC to another and connect to the same server.
So it’s nice to see IDEs getting features previously unique to IPython/Jupyter, but it seems that at least for my usage scenario (see 1st post for details) IDEs are not there yet.
You could use VS code remote (or some other editor with this feature) to develop and make changes to your package/module/jl file, but still do the analysis and plotting using Jupyter (+ Revise.jl ). This way you get the best of both worlds.
EDIT: Also, you could consider using emacs/vim inside a tmux session running on the remote machine in order to have a persistent editor session when you switch machines.
emacs/vim inside a tmux session running on the remote machine
I can imagine emacs being able to keep persisntent plots etc next to corresponding code blocks (however didn’t use those editors myself), but tying everything to a remote session means that without reasonably fast and reliable internet connection it’s impossible to view existing code/results/… In Jupyter obviously one can examine and edit the already loaded page. I often experience poor and dropping connection eg while in transport, and jupyter deals with this pretty well. Also typing delays become very uncomfortable when using remote session like this on mobile connection.
While it is indeed possible to get plots and Jupyter like experience in emacs (for example using emacs-jupyter + org-mode, not sure if plots can work inside a tmux session though), here I was suggesting it only as replacement to the VS code part of the workflow I suggested and not for the Jupyter part.
But yes, I guess that with this setup you would not be able to keep working in the editor if your connection dropped (I’ve heard that using mosh instead of ssh is supposed to somehow help with poor connections but not sure exactly what it provides).
I am using Atom/Juno with “ftp remote edit” plugin, and Julia running in a tmux session on the host workstation. So when I connect to it from different editors, the Julia process stays the same, with its workspace. This works reasonably well for me, and I think covers all the use cases that you need. Specifically for plots, you could prepare them in advance, and then just call display(plt) when you reconnect from a new editor. Plot pane in Atom works over remote connection.
I thought of a similar setup, unfortunately there are several issues with it: a) as I understand, you cannot easily have several independent julia sessions, each automatically corresponding to separate “workspaces”, e.g. directories with source files; b) more importantly, dealing with plots this way doesn’t work across julia restarts, requires to keep references to all the plots, giving them names and not forgetting which is which, and overall is just way too “manual” - compare to notebooks, where all the output (including plots) is unambiguously tied to the code producing it and stored together “forever”.
It sounds as if you really want/need to work with an interactive computational notebook, so why not just continuing working with a Jupyter notebook ?
Again, I personally think that the hybrid workflow is the best here. Do things which are best done in notebooks in Jupyter (plotting, data analysis) and other things (development of packages) in your editor of choice.
Yes, I really do want and need, but the reasons I’m trying to find something else are also outlined in the first post: jupyter really lacks in text editing capabilities - even auto indent often works incorrectly, and in debugging/code navigation.
Sounds to me like what you want is just to do your work via ssh on a remote server, possibly in a docker container. It doesn’t really sound to me like anything you’re doing is unique to Jupyter (full disclosure: I despise Jupyter).
Sounds like you want to run Julia in a docker container, you can do this anywhere, it has nothing to do with Jupyter.
For vim and emacs at least, it is trivial to reproduce a setup, see for example my nvim setup file. I’d imagine that any IDE’s that are any good have a similarly easy to reproduce setup.
This does seem like one area in which JupyterLab may have a slight advantage. You don’t typically need a ton of bandwidth for ssh but you do need some and if you have a bad connection lag can be frustrating.
REPL output doesn’t disappear and even stores a history between sessions in .julia/logs/repl_history.jl.
Nothing is stopping you from running arbitrarily many instances of Julia.
For me personally the fact that Jupyter lab capbilities are limited is actually a feature- it incentivizes me to refactor my code into more organized packages so I can work on it with a normal text editor, instead of keeping everything in huge messy notebooks .
This jupyterlab-lsp looks cool in any case. For jumping to definition there is also this simpler extension (from the same author of the lsp one) which seems to just work.
For what it’s worth, emacs-jupyter has full support for remote kernels–no tmux session necessary. What the op is requesting sounds like it would be perfectly served by emacs-jupyter + org-mode + tramp, but I’m hesitant to suggest jumping ship to emacs for this kind of setup because there are so many interconnected moving parts that getting it working reliably might be a bit fiddly for somebody new to the editor.
But @aplavin, for what it’s worth, emacs-jupyter + org-mode + tramp satisfies every single point in your original post.
?