OK, then use Pluto. There is a reason why no other Data Scientists do. Anyway, good luck with it!
I’d say it is not dependent on who you are but on what task you want to do. For quick / few steps tasks/a well-developed notebook, Pluto fits well. But for a task that consists of many sequential time-consuming processes, especially during the development phase, it would be better to use Neptune since it is designed to avoid such time-consuming unnecessary re-evaluation of all subsequent cells.
That makes sense. The approach that I was thinking of, was to compartmentalize parts of a program. I think COBOL does a good job of this with paragraphs.
This is similar to what @Mason points out here
I am very sympathetic to this view as well. Reactivity is great for avoiding mistakes where you change a variable (or a function) somewhere upstream and forget to re-run cells that depend on that variable (or function). For me, though and my personal style of development, the tediousness of having to wait very long for changes to propagate through the notebook outweighs the benefits.
You choose. But where the search for “neptune notebook data science” si going to lead everyone is certain.
For Julia packages, I always use PackageName.jl
to search for it.
The first hit when googling “neptune.jl” is this Discourse thread.
Kuiper belt sounds nice and Pluto navigates in that space.
Hi All. Neptune.jl has only been in existence for a couple of months, and only in the Registry for just over a week. Knowing Google (and the level of interest), I think it will be searchable soon.
In all fairness, if you post this, you should also post your last message - the one that led to his reply. As a side note, I had the pleasure of talking to Fons about Pluto and it was a great interaction.
Also, Julia makes it very easy to compose packages, so one could, at least in theory, build an extension package, instead of having to make PRs to add features to existing packages.
But anyway, don’t want to linger on the divisive parts of this thread. Just giving my 2c. Congratulations on the release!
You could start Pluto with
Pluto.run(run_notebook_on_load=false)
to avoid this.
Regarding the execution barrier proposed in Feature idea: a temporary execution barrier · fonsp/Pluto.jl · Discussion #298 · GitHub, I think this would greatly help to mitigate issues with long-running cells (which I also frequently encounter) while keeping the advantages of reactivity. As far as I can see in this Issue, @fonsp is quite positive to it, just someone has to implement it
In general I think reactivity is a huge advantage: I recently “inherited” an old Jupyter notebook and struggeled to reproduce his results because the cells were exectuted in a non-linear order and changed after execution. By design, this is avoided in Pluto.
This can be removed with @Mason 's PR. Is this something that could be added to Pluto as well? Would be amazing <3
Is PlutoUI plug-and-play in neptune? I would assume so, if it is based on Pluto - that would be sweet.
Regarding the name conflict with Neptune.ai, I was thinking Helium.jl because it is a lightweight noble gass
No, the 1 expression per cell thing is by design in Pluto and they don’t intend to change it.
Really? I mean, it just looks terrible - like a duck-tape fix. It is one of my largest issues with pluto…
(Which I just want to state that I am a huge fan of - I generally love the interactivity as well, but would appriciate it being optional.)
There’s a lot of discussions of this in the Pluto.jl github repo.
My understanding is that one of the bigger considerations is that large, compound expressions are very difficult to analyze for their dependancy tree. This means there’s more potential for the reactive parts of Pluto to not understand what cells to re-run when things change.
Hence, they aim to discourage people from writing hard to analyze code by making it uglier with the begin
/ end
thing.
Just my two cents. I had some limited experience with Pluto Julia internals - I feel a lot of them should be made into a package instead of staying together in a huge package - the server backend, the evaluation logic, dependency analysis etc. are quite independent.
Once these things are split out, you can replace parts by parts and implement your favorite thing, not just notebooks but more generally, web-based UI with a Julia backend.
This would be very useful for a lot of things since even notebooks themselves are not always the perfect form of demonstrating things. But Pluto provides a native way to evaluate a piece of Julia code from the webpage and change the corresponding web content - I think this is where it is highly valuable and should be generalized.
This is kinda different from writing web frameworks like Genie - you can edit the web components directly from the website, so it can be partially or fully reactive, with some tweaks in the backend. It is more similar to programming things via JavaScript in a browser directly.
“What a remarkable idea. I am tired of writing begin…end stuff, which is not only tedious but has an extra unpleasant indent.”
In case you haven’t followed below, this should work now
I haven’t used pluto, but I have generally avoided notebooks for the reasons that pluto seems to solve. Eg I don’t like how cells can have arbitary unknown relationships and you can get in weird states. What I don’t get is why Pluto is worse about re-running long running cells if it does dependency analysis. Isn’t the whole point that it re-runs cells only if a cell they depend on has changed or they themselves have changed? In that case, why are you getting undesired re-runs?
Sometimes I’ll forget that the long-running cell depends on some upstream parameter I want to tweak, or I’ll forget to manually put in a PlutoUI button as an execution barrier before updating the parameter. On Windows, there’s no way to gracefully interrupt execution, so I’ll just sit there, waiting for the wheel to finish spinning, or I’ll close the notebook, comment out the offending cell, and re-open.
edit: Or I’ll discover that I need another package, so I’ll add it to my using
cell, but any existing cells that depend on the already-included package names get marked as stale and are re-evaluated.