There’s not a lot of resources or documentation, to be honest. But do feel free to ask specific questions here or elsewhere.
CSTParser is in very good shape performance- and functionality-wise, and it’s quite well battle tested due to being part of the VSCode extension, among other things. CSTParser is also a lot faster than the flisp parser — 20x or so when I measured it. Another great thing about CSTParser is that it produces precise source code location info, due to its use of a CST rather than an AST.
As of the start of this year, I thought CSTParser could improve in several areas:
- Documentation - there’s almost none right now and I found the structure of the code a bit of a mystery.
- Error states - CSTParser knows fairly precisely where an error occurred, but the reason for the error can be hard to extract.
@zacln or @davidanthoff might be able to provide guidance or explain where I’m mistaken about the state of things
I think the advantages of rewriting the compiler frontend (parsing+lowering) would be to:
- Preserve more source information to provide precise compiler diagnostics. For a start, syntax errors could be more precise — this is the low-hanging fruit.
- Expose some of the source code transformations and analyses from lowering as APIs. This would enable certain types of advanced metaprogramming which currently require macros to re-implement parts of lowering.
- Make the compiler implementation more accessible to the julia community.
- Possibly performance in some circumstances. I don’t think the compiler frontend is a huge bottleneck compared to inference and code generation in the default configuration. However in certain circumstances (eg, low optimization levels) the difference could be substantial. Largely this is just my gut feeling — should be measured.