Compiler tools and workflow for compiler optimization

Congrats on taking the plunge! The good news is that many people have gone through the same journey. The bad news is that most of us are stuck at roughly the same place you are :sweat_smile:. I know you replied on the thread, but there’s a reason Why are compiler devs in such high demand for Julia? - #10 by jpsamaroo is so important!

Until then, the answer depends on what you want to do. Working with untyped IR (specifically CodeInfo) is better documented. IRTools lets you go from that to a custom IR which is arguably easier to learn, but I wouldn’t recommend it for anything other than learning for various reasons. Here I’ll also plug @Tomas_Pevny’s great course materials covering both. They’re a rare bit of polished documentation on the subject. Other ways to glean information include reading through the source/docs of libraries such as Cassette.jl.

Working with typed IR (CodeInfo or IRCode) is a lot tougher. The current approach employed is to use AbstractInterpreter’s + opaque closures to capture function IR and feed transformed IR back into the compiler respectively. A few libraries have tried this approach, but because the APIs are so new they have either bitrotted or are not a great intro to the topic. One set of keywords which may help your search is “compiler plugins”. These were proposed a couple years back as a more user-friendly and stable interface for working with the compiler, so some of the design docs and draft PRs. Unfortunately the work stalled a little while back so I would treat these more as documentation than ready-made solutions for you.