I have been mildly obsessed with the Thompson’s construction algorithm and recently implemented a version of it in julia. I also wrote a couple blog posts for additional context. This code works but feels like it could be more idiomatic if only I grokked julia metaprogamming better. Specifically, I am constructing a collection of matching fragments as a Dict{Symbol,Tuple{Expr,Vararg{Expr}}} so I can convert them to separate functions and implement the regexp matching as an interpreter (see function match) or inline them into a specialized matcher (see function jit). How can I get rid of exprs altogether, define char!, posstar!, etc directly as functions yet still access their AST to inline them in a specialized matcher?
Disclaimer: I briefly looked at your code but it’s a bit too complex to understand in a couple of minutes. I think I have a highlevel understanding of how it works though.
IIUC, you have a dict with functionname->body from which you either generate Julia function to use in the “interpreter mode” or you, in “compiler mode”, stitch them together into a single function expression for Julia to compile. The latter part you don’t like. While I personally think that approach is fine, you probably don’t need to inline yourself. You can simply tell Julia to inline the function (by adding @inline to where you define the functions) and then you don’t need to handle the function bodies as expressions anymore.
To give you a better picture of what “metaprogramming with macros” is: Think of it like syntactic sugar. If you notice you need to write somewhat repetitive code with a clear pattern but much less “actual data” then there might be room for a macro. When trying to design a macro after having identified such a repetively, well-structured code fragment, I usually try to write down what I actually would like to write instead (that is still valid Julia code). Then the macro just needs to translate what I want to write into what I need to write. And that’s the essence of what macros are and do. I don’t really a good macro opportunity for your code here, except maybe doing a string macro like Julia’s regex-strings, that takes a pattern string and replaces it with the matching function. This macro could be written very little effort given your current codebase.
Thank you abraemer for taking the time to look at my code, however briefly. Thank you to the entire community as well for giving me this rubberducking opportunity. Explaining what I didn’t like about my code crystallized how I could improve it. I didn’t like how I dissected simple functions because I couldn’t extract their body as an AST. Quoting them was the obvious work-around but I approached it the wrong way. I didn’t need to rewrite every function as quoted body and args then generate them later. Instead, I can quote the entire chunk of code where those functions are defined so I can evaluate it and access their bodies by walking the AST. 0e00457 looks much more like e3e5bc7 with
matchers = quote
# all function definitions
end
exprs = Dict{Symbol,Tuple{Expr,Vararg{Expr}}}(
expr.args[1].args[1] => (expr.args[2], expr.args[1].args[2:end-4]...)
for expr in matchers.args if is_expr(expr, :function)
)
eval(matchers)
I like this better because it’s so much less intrusive.