[ANN] ExpressionExplorer.jl and PlutoDependencyExplorer.jl – the internal reactivity algorithm of Pluto!

We just released two new packages! :tada:

Both packages are quite small and load fast, with minimal dependencies. The packages were part of Pluto’s source code, but we decided to release them separately so that they can be used in other projects!

ExpressionExplorer

Find all variables referenced and defined in an expression. This package is used internally by Pluto to find links between cells.

Quick example

julia> using ExpressionExplorer

julia> ex = :(const words = split(line));

julia> node = ExpressionExplorer.compute_reactive_node(ex);

julia> node.references
Set{Symbol} with 2 elements:
  :line
  :split

julia> node.definitions
Set{Symbol} with 1 element:
  :words

For more information, check out the documentation: ExpressionExplorer.jl

PlutoDependencyExplorer

This package contains Pluto’s dependency sorting algorithm. Given a list of cell codes, PlutoDependencyExplorer can tell you in which order these cells should run. For example:

julia> import PlutoDependencyExplorer as PDE

julia> struct SimpleCell <: PDE.AbstractCell
           code
       end

julia> notebook = SimpleCell.([
           "x + y"
           "x = 1"
           "y = x + 2"
       ]);

julia> empty_topology = PDE.NotebookTopology{SimpleCell}();

julia> topology = PDE.updated_topology(
           empty_topology,
           notebook, notebook;
           get_code_str = c -> c.code,
           get_code_expr = c -> Meta.parse(c.code),
       );

julia> order = PDE.topological_order(topology);

julia> order.runnable
3-element Vector{SimpleCell}:
 SimpleCell("x = 1")
 SimpleCell("y = x + 2")
 SimpleCell("x + y")

For more information, check out the documentation: PlutoDependencyExplorer.jl

Which package should I use?

PlutoDependencyExplorer uses the low-level package ExpressionExplorer to find the assignments and references of each cell. PlutoDependencyExplorer uses this information to build a dependency graph between cells (i.e. a NotebookTopology), which can be used to find the order to run them in (a TopologicalOrder).

If you are interested in ordering a list of expressions in execution order (the order that Pluto runs cells in), then use PlutoDependencyExplorer. If you just want to know which variables are assigned or referenced in a single expression, use ExpressionExplorer.

Let us know what you think!

We hope that these packages can be useful to the Julia community! Do you have any fun ideas? Or comments about the package? Let us know!

36 Likes

Very cool, thanks for this - I wonder if it could be used to implement a Pluto-like system for Julia in emacs…

With a bit of work, yes. I expect based on the people I know interested in this and time/current projects that we’ll see a very high quality ob-julia in about 1-2y.

2 Likes

Thank you!!

This is amazing! I can think of several applications for GenAI/Agents to help them understand the code!

2 Likes

Nice! Please let us know if this package is useful or if something is unclear!

We always value feedback, not just bug reports!

It was a great to help me clarify what I need + more robust than my simple solution written by GPT4.

But I had to hack around it to get the information I needed.

Would it be simpler to mention my use case here?