ANN: Julia parser for JavaScript based on lezer grammar v1.0.0

Package Announcements
, ,
1

Hi! :waving_hand:

The Pluto team is proud to announce @plutojl/lezer-julia version 1.0! This is a parser for Julia syntax to be used in JavaScript: you can use it in web browsers, NodeJS, Bun or Deno.

Our parser is of very high quality: it parses even the most esoteric Julia code correctly, and it has a detailed parse tree, which makes pattern matching easy.

What is a JavaScript parser for Julia?

This package gives a lezer grammar, which you can use to parse Julia code in JavaScript. This is useful for interactive web experiences with Julia. You can run this snippet in Deno:

> import { parser } from "npm:@plutojl/lezer-julia";

> const code = `function area(r)
    return Ļ€ * r^2
end`;

> const tree = parser.parse(code);

> console.log(tree.toString())
Program(FunctionDefinition(function,Signature(CallExpression(Identifier,Arguments("(",Identifier,")"))),ReturnStatement(return,BinaryExpression(Identifier,TimesOp,BinaryExpression(Identifier,PowerOp,IntegerLiteral))),end))

The result is a Concrete Syntax Tree (CST) of your code. You can use this tree object in various ways. You can walk through the tree in a functional style or an imperative style. You can query the tree at a string position, and you can make simple queries.

Incremental parsing and CodeMirror 6

Lezer grammars support incremental parsing, which means that the syntax tree can be constructed once for a large piece of text, and then updated incrementally when you make a small change to the text, without re-parsing the complete document.

Because of this property, lezer is ideal for use in CodeMirror 6, a web-based code editor using in Pluto, Jupyer, GitHub and many more sites. Our package means that CodeMirror 6 now has very good Julia support.

Demo website

I vibe coded a demo website! Here you can see the parser side-by-side with the old regex-based CodeMirror 5 parser. At the bottom, you can click-to-expand the parse tree. Demo site

image
image2518Ɨ1222 407 KB

I made a PR to Jupyter! :sparkles:

I am working on a PR to Jupyter and JupyterLab to use our parser. They already use CodeMirror 6, but for Julia they currently have a simple (unmaintained) regex-based parser. With the new parser, Jupyter would have more rich syntax highlighting for Julia, and it would add support for new Julia syntax features.

Before:

image
image2560Ɨ1426 473 KB

After:

image
image2560Ɨ1354 459 KB

Credits

Iā€™m posting this because I am excited about this parser, but most of the work was done by others, mainly from the Pluto team! The parser was first written by Andrey Popp. Michiel Dral and Paul Berg improved the parser for use in Pluto.jl. Sergio A. Vargas (also main contributor of tree-sitter-julia) did a major rewrite for more complete Julia support, and a more detailed parse tree. I did some recent bug fixing with AI help.

6 Likes