Struggling with meta programming

General Usage
1

Hi all,

I’m attempting to teach myself how to do Julia meta programming. My current exercise is to write a macro that produces a variant of zip, in that you give it a single dimensional array, and it returns an iterator that gives tuples, each of which consists of elements i through i+N of the array.

In other words, iterating through @zipn([1,2,3,4,5], 3) should return, in order, (1,2,3), (2,3,4), (3,4,5).

Here’s my macro. It doesn’t work.

 macro zipn(vec, N)
     expr = Expr(:call, :zip)
     for i=1:N
         push!(expr.args, vec.args[i:end-(N-i)])
     end
     return expr
 end

Specifically, for some reason it returns an Iterator that looks like:

Base.Iterators.Zip{Array{Any,1},Base.Iterators.Zip2{Array{Any,1},Array{Any,1}}}(Any[1, 2, 3, 4], Base.Iterators.Zip2{Array{Any,1},Array{Any,1}}(Any[2, 3, 4, 5], Any[3, 4, 5, 6]))

The thing that confuses me is that the code works like a charm when executed one line at a time in the REPL. It just doesn’t work when it’s inside the body of the macro.

Clearly I’m missing a concept. Any help figuring out what would be appreciated.

2

Here’s the first piece of advice about metaprogramming: usually you don’t need it. In such cases, you are much better off just writing a function

zipn(v::AbstractVector, N::Integer) = (tuple(v[i:(end-N+i)]...) for i ∈ 1:N)

One reason this advice is important is because when you get to things that really do require metaprogramming, it is very important that you first have all the functions you might need to appear in your macro. For example, the very best way to turn the above into a macro would just be

macro zipn(v, N) :(zipn($v, $N)) end

You might say that this is silly, and you’d be right, but this goes to show you that it is never appropriate to write a macro if it can be easily replaced by a regular function.

If you are interested in metaprogramming, here’s an exercise I suggest: suppose I have a Dict{Symbol,Int} and I want to do arithmetic on its elements. You can of course do this by doing e.g. dict[:a] + dict[:b], but if this were something you had to do very often, you might prefer to be able to do

@keymath dict :a + :b

This example is admittedly still a little bit silly, but it is functionality that is important for packages such as Query.jl and DataFramesMeta.jl. I suggest trying to implement this as an exercise: it’s pretty close to a realistic case where macros are necessary.

Also, one disadvantage that Julia has when it comes to metaprogramming is that its AST is very complicated compared to other Lisp variants such as Scheme and Clojure. Julia was built this way quite deliberately: the devs wanted to have metaprogramming but they also wanted a language with a syntax that made it easy to use for other purposes as well. This is why the syntax of Julia usually looks like it more closely resembles Ruby or Python than Lisp. While this is a great feature most of the time, it can sometimes make metaprogramming a headache. Extracting information from expressions usually requires quite a bit of knowledge about Julia’s AST. An important tool for making this much simpler is MacroTools.jl. It’s unfortunate that the main Julia documentation does not, at the very least, make people abundantly aware of this package, which ought to be considered a necessity for doing metaprogramming.

10 Likes
3

What is kinda fun is figuring out how to do this with tuples:

f(a, b, c, d...) = (a, b, c), f(b, c, d...)...
f(args...) = ()
1 Like