[WIP, RFC, ANN] PlutoWorkspaceExplorer: Inspect the variables definined in a Pluto notebook

JackDevine · March 30, 2024, 11:22am

~~PlutoVariableExplorer.jl~~
PlutoWorkspaceExplorer.jl creates a table of the variables in your current Pluto session.

Given my background in MATLAB, this is a feature that I have wanted for a while. Unlike MATLAB (or any other variable explorer that I have ever used), ~~PlutoVariableExplorer.jl~~ PlutoWorkspaceExplorer.jl is aware of the dependencies between your variables. It also lets you jump to a variable definition through the pluto-links functionality.

Before I register the package I wanted to give people a chance to bikeshed the name or suggest an umbrella package that this functionality could be included in. Feel free to leave any suggestions/feedback here or on Github

I’d like to thank @fonsp and others for all their work on Pluto and related tools. If Pluto wasn’t such a high quality product, then this work would be much harder and much less rewarding.

From the package README:

Interactively inspect your workspace values and topology in a Pluto session.

Usage

Create the following cells: ~~(for now you will have to do import Pkg; Pkg.add(url="https://github.com/JackDevine/PlutoVariableExplorer.jl") first)~~

import PlutoWorkspaceExplorer as PWE

@bind _update PWE.update_notebook()

_update; PWE.variable_explorer(PlutoRunner)

If you dont create the _update variable, then the workspace explorer will only be updated when you run the cell that defined it. The _update variable will change every time you press Shift+Enter.

Example view of the variable explorer:

Link to the repo

Some known issues

There are some open issues that I am not sure how to fix:

EDIT I also remembered an issue with how pluto-links are handled

I should be able to come up with solutions to these problems in my own time, but any advice or help would be greatly appreciated

Happy Plutoing and have fun using the variable explorer in your own notebooks!

Guillawme · April 3, 2024, 11:14am

I have missed the workspace explorer too! (but from RStudio). I even recently wrote about it in the feedback field in Pluto, so seeing this post makes me very happy. This package is going to make my life so much easier when writing Pluto notebooks, thank you! Tracking dependencies between objects listed in the inspector and links to the object definitions are two features that make this explorer even better than the equivalents in RStudio and Matlab.

I like the name, on point and descriptive. Or maybe name it PlutoWorkspaceExplorer? Just because it doesn’t only list variables but everything with a name attached.
But I think the term “Prependencies” is confusing. I suggest finding better terms for these two concepts. Maybe “parent objects” and “child objects”? I think these terms make it more obvious where the different objects are relative to each other in the dependency tree.

JackDevine · April 4, 2024, 12:23am

Thanks for your kind words!

… maybe name it PlutoWorkspaceExplorer?

I think the term “Prependencies” is confusing. I suggest finding better terms for these two concepts. Maybe “parent objects” and “child objects”?

Thanks for these naming suggestions, they are really good and I completely agree that they are an improvement over what we have now.

I will make the parent/child change as soon as I can set aside some time.

While I agree that PlutoWorkspaceExplorer is a better name than PlutoVariableExplorer, I will wait a little bit to see if anyone has any other name suggestions, or umbrella projects to suggest before I change the name of the repository, which will break links.

Guillawme · April 4, 2024, 7:15am

Of course, don’t make such renaming decisions lightly.
And glad my suggestions are helpful.

JackDevine · May 25, 2024, 8:15am

Hi all, sorry for the delay. This package has been registered and the naming suggestions above have been applied. That means that you can now try out the workspace explorer with just

import PlutoWorkspaceExplorer as PWE

fonsp · May 27, 2024, 4:36pm

Ohhh so cool!! Very exciting @JackDevine !

Did you see this new Pluto feature? It might be useful!

github.com/fonsp/Pluto.jl

⚡️ AbstractPlutoDingetjes.Display.with_js_link to request calculations and data from Julia dynamically

fonsp:main ← fonsp:AbstractPlutoDingetjes.Display.with_js_link

opened 11:53AM - 21 Nov 23 UTC

fonsp

+858 -42

## NOTE: this PR is merged but not yet released! Use `Pluto#main` to try it out,… and please let us know how it went! Good feedback is also welcome :) New JavaScript API to request calculations and data from Julia dynamically. This is useful for: - Lazy loading more data in a display - An infinite zooming mandelbrot display - A table with millions of rows, that loads data on demand - Letting Julia calculate something on-demand, without requiring a `@bind` interaction - A `@bind` input widget that lets you type Julia code with a submit button, but it validates your input while you type. - An interaction where you select two points on a map, and Julia calculates the shortest path between them, and they are displayed *on the same map*. - Play chess against an AI written in Julia. Matching AbstractPlutoDingetjes PR: https://github.com/JuliaPluto/AbstractPlutoDingetjes.jl/pull/13 Supersedes #991 and #2392 # API The easiest way to use this API is with HypertextLiteral. The use is very similar to `APD.Display.published_to_js`: the new function `AbstractPlutoDingetjes.Display.with_js_link` returns a "piece of JavaScript code" that you interpolate directly into a `<script>` tag. In JavaScript, the "piece of JavaScript code" returns a function. You can call this function with an argument (which will be passed to your Julia function), and it returns a Promise that resolves to the answer from your Julia function. ```julia @htl(""" <script> const sqrt_from_julia = $(AbstractPlutoDingetjes.Display.with_js_link(sqrt)) // I can now call sqrt_from_julia like a JavaScript function. It returns a Promise: const result = await sqrt_from_julia(9.0) console.log(result) </script> """) ``` # Serialization and inner workings The request and response use Pluto's internal communication protocol (with MsgPack and WebSocket), so the performance is optimal. The same serialization is used as `published_to_js`, so in particular, `Vector{Float64}` or `Vector{UInt8}` are really fast. Since this API is designed for one-off requests, this communication does not go through Pluto's state management (the request and response are not stored in the state). If multiple clients are connected in parallel, then the messages are not shared between clients. # When not to use it This API is only meant to support use cases that can not be covered with `@bind` and `Display.published_to_js`. If possible, the use of these APIs is preferred over `with_js_link` because they will work with the Static HTML export and PlutoSliderServer. # Example <img width="730" alt="image" src="https://github.com/JuliaPluto/AbstractPlutoDingetjes.jl/assets/6933510/1dcd30c9-1be4-4edb-9f55-0129f436c2fd"> <details> <summary>Code</summary> ```julia @htl(""" <div> <p>Input:<br> <input> <input type="submit"></p> <p>Output:<br> <textarea cols=40 rows=5></textarea> <script> let sqrt_with_julia = $(AbstractPlutoDingetjes.Display.with_js_link(sqrt)) let wrapper = currentScript.closest("div") let input = wrapper.querySelector("input") let submit = wrapper.querySelector("input[type='submit']") let output = wrapper.querySelector("textarea") submit.addEventListener("click", async () => { let result = await sqrt_with_julia(Number(input.value)) output.innerText = result }) </script> </div> """) ``` </details> # TODO - [x] Exception handling inside the callback - [x] Logging inside the callback - [x] Stdout inside the callback? - [x] Running the callback async or in a thread? ~Right now the callback runs `@async` on the web server.~ Right now it runs sequentially with other code execution. But then lots of calls can clog up the queue... And queued messages can be invalid once they get executed. - [x] What should happen when the notebook is running while using the callback? What if the cell itself is running? - [ ] What happens when the callback takes long? - [x] Make sure that everything is cleaned up properly - [x] Tests - [x] Docs - [x] Future-proof to allow a bi-directional API in the future. EDIT: with the invalidation callback it can be done! Need to make a MWE and maybe test it - [x] Could be `published_to_js(::Function)`... EDIT: that would be cute but then `is_supported_by_display` can't check for it soooooo no :( - [x] Add an invallidation callback (I already have this on my computer but still need to push) - [ ] What about PlutoSliderServer? Maybe also a `pure::Bool` kwarg to declare that your callback is pure? Or maybe that should be assumed by default? (Yes because multiple clients can connect to the same notebook) How do you declare the possible values etc? - [x] Disabled cells should have js links disabled?

JackDevine · May 28, 2024, 11:58am

Thanks @fonsp!

I did have a look at AbstractPlutoDingetjes, because I thought that it would allow me to setup the workspace explorer in one cell.

But I wasn’t able to get it to work because the workpsace_explorer function returns a HypertextLiteral.Result which is a table of Julia data structures (so the variable explorer can use Pluto’s display methods). If I understand AbstractPlutoDingetjes.Display.with_js_link correctly then the returned value would have to be a data structure that javascript can work with (and then I would have to have my own custom javascript display code).