I wander why so many people seem to prefer using a notebook, i.e. the Jupiter Notebook, to develop Julia code, and I am curious to understand which is the main advantage over using a more classical IDE approach (Atom/VsCode on *.jl files), given that using a classical IDE you can visualize plots, inspect variables or tabular data, debug, profile, choose to run the code by line, by block, by cell…
I may understand to produce documentation/courses (even if even for that I prefer the literate approach), but for development? I feel I am missing some clever workflow…
5 Likes
Regardless of what the purpose is, small scripts that put together a few packages can more easily benefit from having nicely formatted graphics and text that present the code and vice versa. You can do the same in more static documents, but there is also a benefit to having interactive runnable code instead of pasted text. Runnable code in documents is less beneficial when there are large swaths of code not worth presenting, and the most you’d want it to show up in a notebook is an import statement or a collapsed cell. I find it useful for presenting small experiments with candidate packages in one file.
Another major reason notebooks caught on is it can be a lot easier to change an interactive program in a notebook than in a REPL. With a REPL, you might have to spam the up button or scroll past many similar evaluations to see previously entered code; with a script, you can more easily find a section and make edits, but evaluating everything again is a waste of time (though sometimes a fresh run is necessary to smoothly make widespread changes or make the state of a fresh run). A notebook has aspects of both, and small cells can be run instead of the whole thing. Again, notebooks don’t scale to larger codebases, but we have better options sharing some features when we don’t need to present everything: IDE extensions as mentioned, Revise.jl to link source changes to small evaluations we could otherwise do in the REPL. I don’t think there’s anything you’re missing, it’s just an option you might not need.
2 Likes
I use vim for coding and jupyter + jupytext to do exploratory analysis which I may leave as documentation of what I have done.
2 Likes
I think of a notebook more like an enhanced REPL/script. I can ship a notebook to someone else, or host it online, and they can run it. I can work partway through and easily restart to get the state I want, which is harder to do in REPL or IDE.
1 Like
I usually use Pluto.jl for drafting code. What I like most about Pluto is the interactivity and flexibility. Main advantages:
- I can change structs easily
- I can write some tests for the code and they are reevaluated upon change of definition (and I can have nice visuals)
- Easy to add packages just used for verification/evaluation
Then I copy it to a package once I am satisfied with the code.
1 Like
I actually don’t recommend notebooks for developing code, at least nothing beyond the very early exploratory phases. My recommendation is “if you have been working on a notebook for a week, its time to separate it into a source code library content and a script file.” I give my two cents in more detail during the good scientific workshop:
notice the timestamp, linking to 3:51:48 time in the video.
14 Likes
For me, a killer feature in Pluto is reactivity. Instant feedback on code changes is what I miss the most when switching from notebooks to vscode (be it regular julia code in packages or line-by-line execution in a script). It’s so easy to get used to everything (outputs, plots, tests, …) updating instantly after every code change (:
4 Likes
I like notebooks (Pluto.jl mostly), not so much for developing code, but rather for investigating the behaviour of models/functions. I can get interactive plots with sliders and buttons that let me see how changing parameters changes outputs. Incrementally changing things works much more dynamically in a notebook than at the REPL.
There’s a talk on Youtube called “I don’t like notebooks”. It’s written from the perspective of Python and Jupyter (rather than Julia and Pluto), but it’s worth a watch.
3 Likes
I use notebooks for exploratory work and debugging. What I like over classical IDEs in this setting is the alternation of input and output which persists on saving, making it nice to come back and read. This does require discipline in cell execution order though.
A particular workflow I use when optimising a function is to gradually change the function, with the new version in each cell and the benchmark result as the cell output. Then I can go back and see what the performance considerations were when I inevitably forget.
I also like running the notebook from within VS Code, as you get all the familiar shortcuts and features from non-notebook programming.
I use notebooks for reproducibility in books and papers. I use them for anything more than a small number of plots/tables. For example
and another paper
As soon as the computational results get messy, I start making a notebook.
I do my development in vi+REPL. I use notebooks to make it easier for my readers to duplicate my results.
3 Likes
I love the fact about notebooks that plots are persistent, its very easy to keep everything organized, because the plot tells you usually what each individual cell is used for and you dont need to think so much on which plots you save as you do with a REPL based approach.
I really like notebooks conceptually but somehow havent really made them work for me in julia.
One of my biggest gripes I have is the lack of a repl (is there any notebook that has one?) I find myself often manipulating the project environment to install or remove new packages, and import Pkg; Pkg.add("...") feels just way more cumbersome than typing ] add .... I also have to create a new cell and delete it directly after every time because I dont want to mess up my notebook.
Also, at least in VSCode i find it hard to navigate through to a package that im developing. In a normal VSCode cell, I usually can ctrl click on the function and get directed to the definition. Notebooks seem a bit too cell oriented for me, when i do ctrl+z, it only affects the currently selected cell which feels weird.
So yeah, I think notebooks do a fantastic job at making a nice framework for plotting things and for documenting. For me here are only some tiny features missing for me to properly enjoy them.
Edit: im mostly talking about jupyter notebooks, i have not really used the other ones but Im note sure there is one that doesn’t share the characteristics i mentioned
2 Likes
To me, a symptom of the different approaches can be seen in the choice of words.
I wander why so many […] prefer […] a notebook […] to develop Julia code,
I do not develop code. I develop a computation (or a set of computations).
The computation I develop is written in text (latex or markdown), equations (latex) and code (julia), and is visualized in equations and the resulting plots.
If you think of your work as developing code, then I think your classic IDE enhanced with a few notebook-like features (running individual lines and seeing plots) is fine.
When I develop my computation, I prefer a notebook (with a few IDE-like features, included in jupyterlab).
As part of my work, I am creating a julia package using notebooks, with the package consisting of several modules, and each module being developed as a notebook (using jupyterlab + jupytext). As that code started as a “computation” and has transitioned towards being a “codebase”, I think that, with hindsight, it might have been useful to switch from a notebook to a classic set of files at some point.
Nowadays, I mostly use the package-as-a-notebook as a literate programming tool.
(As a side remark, I use jupyter instead of pluto because my computations take a long time to run, and I expect putting a 12-hour computation in pluto not to work well.)
5 Likes
I think you start with a wrong premise assuming that “so many people prefer using notebook to develop julia code”. I’m sure if you ask the julia developers, the main IDE will be vim/emacs/vscode for code development and not jupyter notebook. Can you provide statistics to show this preference? I really doubt you can create and maintain julia package using notebooks. This is a a plain wrong assumption. Notebooks are good for presentations, demo, explorations but not really good for code/package development in julia or in any language.
2 Likes