(Not-So-)Crazy idea: Make the julia docs a Pluto notebook

How about adding Pluto export support to Literate.jl
1. Introduction · Literate.jl?

1 Like

It’s already in the works AFAICT:

https://github.com/fredrikekre/Literate.jl/pull/120

3 Likes

Mathematica documentation has all of the code examples fully interactive. I have found this to be super helpful, as it allows me to try variations on the examples without lots of copying and pasting.

4 Likes

Tadaaa it’s done:

https://julia-docs-pluto.netlify.app/

26 Likes

Very cool. This is also a good demonstration of how hard this project would actually be to do well, requiring a lot of renaming and reworking of some examples.

I actually think most of the basic rewrites can be automated:

  • deduplicate variable definitions by numbering them (x1, x2), and for references, use the most recent number
  • find cells that use display, println etc and apply the usual pluto tricks: Text and with_terminal
  • find @ref links and resolve them

But the really interesting thing is (re)designing the material with the medium (Pluto) in mind, and that could be a larger project!

8 Likes

What is the difference between displaying in normal docs and displaying in uneditable Pluto pages? I clicked the “run in Binder” button but it’s still loading after a few minutes so I’m not sure if it eventually works.

There was a binder issue, it’s fixed now, try again! It’s early days for these features :upside_down_face: If binder is in a good mood, you can have a live notebook in 30 seconds

1 Like

As others pointed out already, I think that overall this is an interesting idea but the Pluto layer with Pluto-specific behavior is a major downside. New users going to the official docs of the language should see vanilla Julia working, including the possibility of repeated variable names, default printing behavior, etc. Pluto adds a lot of opinionated stuff on top of the Julia experience that some users may find distracting or even undesired.

19 Likes

In particular, reactivity seems unnecessary in the Julia docs and, IMHO, will most likely lead to a lot of confusion.

4 Likes

(deleted)

The Julia discourse is a tough crowd to please :frowning:

Although my proof-of-concept has a lot of problems when ported without modifications, I am very interested to explore what Julia base docs could look like if it were redesigned with the medium in mind.

I completely agree with the premise of this thread: the computational thinking website shows how engaging your documentation can be if you wrote it in a reactive environment, with the intention of being interactive and runnable by the reader. Let’s use that to introduce people to Julia! Let’s make Julia more fun to learn! What if Julia’s docs were so cool that people would share it with their friends?

16 Likes

Do we want people with friends involved?

14 Likes

That’s the spirit! I believe what’s sorely missing right now is a short and condensed introduction to the language as whole (e.g. like A Swift Tour for Swift). One self-contained possibly reactive document…

EDIT: The medium is the message :wink:

7 Likes

Is it a likely proposition to have an online Julia that is not super laggy? Even 30 seconds is quite a lot for websites. I like the basic idea of having beginner docs that are conceived as an interactive experience, but Julia is not an easy target for a snappy online app.

Maybe it could work to tell people to start a local Julia, and then connect the website example with that kernel somehow? That would filter out people who just want to hack around a little bit without any setup, though.

1 Like

As long as it is optional, I can agree with your sentiment. However, one thing I love about Julia is that when everything is running, it is amazingly fast. If you ask me, fast things are cool. I typically work on the REPL, with the i3 window manager and Vim. Things respond instantly even though I’m not spending much on my computers these days (assuming that Julia has compiled everything, of course). This is the same for the Julia documentation. I’m now visiting my parents who have a 750 kB/s download speed. (Note that these kinds of speeds, or worse speeds, still exist in many places.) Here, reloading the strings page from docs.julialang.org takes less than a second on desktop and mobile. On the other hand, a basic page (abstractions) from computational thinking takes 4 seconds to reload! On my phone, I had to reload a few times to see something and the Safari reader view doesn’t work. When reading docs, I often go back and from between different pages which would be slowed down considerably by these kinds of loading times.

So, I understand that some people would like to have dynamic pages, but please do keep people in mind who have lower-end hardware, slow internet connections or just want to quickly find some information.

11 Likes

I have no doubt that we could have some beautiful, functional, nice docs through Pluto, but I also can’t help but feel that @juliohm’s point really resonates with me too.

The julia docs are where people go to learn about julia, so presenting that code in a medium that has a restricted subset of the language semantics (even if that subset is quite powerful and useful) is perhaps not a good fit for the language documentation itself.

That said, it doesn’t have to be an all or nothing thing. Perhaps there are sections of the docs that would benefit especially well from reactivity without confusing users with different semantics. Maybe those sections are a good fit for Pluto integration.

3 Likes

Having very excitedly jumped on the first post, I think the spirit of Pluto is more important than the actual implementation here. That is, encouraging documentation that is more like a walkthrough than a brief blurb + reference and judicious (but not over the top) usage of interactivity.

Much virtual ink has been spilled on this forum about the Divio docs system. Though I’m wary of taking their model as gospel (some more empirical evidence would be welcome), I agree that this Pluto idea (though again, not necessarily with Pluto) would hit quadrants that the current docs do not. I also think they would allow the current docs to become more focused and geared towards e.g. reference. Currently there are a few areas which straddle the boundaries between these documentation types and consequently feel somewhat unsatisfying as either, but are difficult to reform because of how much structural disruption would be required.

4 Likes

Slightly off topic, but I think this lag is more of a binder issue (in that AFAIU it does the complete setup every time you load the page). On the contrary, a julia server helps in fixing the latency problem. If you were to set-up julia on a server yourself (with say HTTP.jl), you could mostly hide the startup time to users by simply doing a few “warmup calls” when starting the server. (The julia process would then be the same across users / sessions.)

On the flip side, having to manage servers yourself is much more annoying than just setting up binder. There may be some “in-between” alternatives that do most of the work for you but would give a reasonably snappy experience. Heroku is probably worth a shot, see eg Deploying to Heroku with Buildpacks · Genie - The Highly Productive Julia Web Framework (genieframework.github.io).

I advocate for separating the concerns of defining the language (and base), and being a beginner tutorial. The current manual is partly both, and for me some parts are harder than necessary because of that. (I mean Base and Standard library, which are lists of functions.)

For DataFrames.jl there are these, which is excellent.

https://dataframes.juliadata.org/stable/
https://github.com/bkamins/Julia-DataFrames-Tutorial

3 Likes