Is there a better way to write this macro?

General Usage
, ,
21

After consulting the code for @eval, it only takes a small tweak to add interpolation to @sexpr.

The macro is now

macro sexpr(expr)
    let transform = postwalk(sexpr_walk, expr)
        return Expr(:escape, Expr(:quote, transform))
    end
end

So that this works:

julia> a = [1,2,3]
3-element Vector{Int64}:
 1
 2
 3

julia> @sexpr (call, +, $(a[2]), 5)
:(2 + 5)

With this a sort of hybrid of the two suggestions can be written as follows:

macro dispatchinstruction(_inst, _vm)
    let inst = esc(_inst), vm = esc(_vm)
        conditions = [:(inst.op == $op) for op in instances(Opcode)]
        actions = [:($(Symbol("on"*string(op)))($inst, $vm)) for op in instances(Opcode)]
        ifdef = @sexpr (:if, $(conditions[1]), $(actions[1]))
        list = ifdef
        for i in 2:lastindex(conditions)
            local e = @sexpr (:elseif, $(conditions[i]), $(actions[i]))
            cons(list, e)
            list = e
        end
        return ifdef
    end
end

Admittedly, in this particular macro, the interpolations make using @sexr a bit noisier than using Expr directly, but that wouldn’t always be true. Besides, this was an experiment to see if I could get it to work.

The “Lisp Maximalist” way to write the loop would be

cons(list, @sexpr (:elseif, $(conditions[i]), $(actions[i])))
list = cdr(list)

But that doesn’t work because Exprs aren’t actually made up of cons cells. Which is fine, there’s no reason they should be.

I think this is still (barely) topical for “better way to write this macro”, but it’s drifting toward the edge, so this is a good place for me to call it quits. Thanks again, to those of you who contributed to the answer.

22

Mason’s @s macro keeps symbols written with : (which wraps a symbol with an expression layer but the inner interpolation unwraps that), so the effect of $-interpolation in quotes and some source-transforming macros is accomplished by writing without : like in Expr calls e.g. :($(x[1]) + y) == Expr(:call, :+, x[1], :y) == (@s :call :+ x[1] :y). If you can stomach writing : for all the symbols, you can also avoid writing $ for an interpolation effect; it’ll also be more consistent with the keywords that the parser forces you to write with :. Extra typing either way but something must distinguish a symbol or an evaluated instance in an Expr.

23

It does that already, I only unwrap QuoteNode in the head, because a) need to for keywords and b) they aren’t valid there so it’s harmless.

julia> @sexpr (call, q, 1, :symbol)
:(q(1, :symbol))

julia> @sexpr (call, q, 1, symbol)
:(q(1, symbol))

Using Expr needs the extra level of quote:

julia> Expr(:call, :q, 1, :(:symbol))
:(q(1, :symbol))

It’s a tradeoff, but I reckon most Exprs I’d write would be a bit cleaner with the macro, and Expr is always there as an option. Most interpolation would just be @sexpr (foo, "bar", $baz), a full expression needs the parens, but I don’t mind.

I don’t think it should be added to the library for the next release or anything, probably won’t even make a package out of it. Just a fun exercise at this point.

24

You’re not talking about the same things I was, note the absence of : inside my example quote :($(x[1]) + y) and the absence of nested quotes in my example Expr(:call, :+, x[1], :y).

You could distribute a small package, should be stable enough to leave it alone for a while. The dependency MacroTools.jl is technically v0 but it’s been v0.5 for almost 5 years.

25

Many such cases.

These are all just ways to write the same thing. I’d rather not repeat @s in every level of nesting, but his is an impressively functional one-liner.

I tweaked the logic a bit so it’s mix-and-match with quote syntax:

julia> @sexpr (:if, :(b < 5))
:($(Expr(:if, :(b < 5))))

julia> cons(ans, :(return b))
:(if b < 5
      return b
  end)

So that’s nice.

Well, and that’s the thing: until I changed it, there was a bug, I wouldn’t consider the previous behavior reasonable:

julia> @sexpr (:if, :(b < 5))
:($(Expr(:if, :($(Expr(:quote, :(b < 5)))))))

This is not a useful expression to return from that input.

There are a lot of potential interactions and I’m quite sure they would turn up more bugs. If I end up using it enough to decide that it’s actually functional I may as well package it up, no idea if that will end up happening or not.

26

I’d just write the macro like this:

using MacroTools: postwalk
using Base: isexpr

macro sexpr(expr)
    esc(postwalk(ex -> isexpr(ex, :tuple) ? :(Expr($(ex.args...),)) : ex, expr))
end

and then you don’t have to worry about any interpolation syntax or whatever.

julia> @sexpr (:if, (:call, :(<), :x, length([1,2,3,4,5])), 
               (:call, :+, [1 2 ; 3 4], :b),
               (:call, :-, [5 6 ; 7 8], :c))
:(if x < 5
      [1 2; 3 4] + b
  else
      [5 6; 7 8] - c
  end)
3 Likes
27

I see that as a good example of the more complex version making the right tradeoffs:

julia> @sexpr (:if, (call, <, x, $(length([1,2,3,4,5]))),
                      (call, +, [1 2 ; 3 4], b),
                      (call, -, [5 6 ; 7 8], c))
:(if x < 5
      [1 2; 3 4] + b
  else
      [5 6; 7 8] - c
  end)

It’s a good tradeoff to get bare symbols, this has the same number of parens and much fewer prefixes. An interpolation-heavy workload would be noisier, and I’d just use Expr syntax in that case.

But I haven’t really used it in complex situations, and I bet there are some remaining cases to work out.

28

A post was split to a new topic: Want to start leaning

29

Note that it’s possible — and useful! — to plop actual functions directly into the AST. So Expr(:call, +, 1, 2) would always use the + function that was defined when you created the expr instead of resolving to whatever + happens to be when the Expr is actually evaluated.

1 Like
30

What a funny coincidence!

I literally just added a horrifying hack to get unescaping semantics.

julia> module M
          using ..Main
          unescaped(a) = a
          sx = @sexpr (:if, :(2 < 3), (call, `unescaped`, 'a'))
       end
julia> M.sx
:(if 2 < 3
      Main.M.unescaped('a')
  end)

Those of you of a Lispy temperament may recognize the use of backquote here :wink: have to match them though, different parser and all.

You don’t want to see my hack, it involves Base.eval and two calls to Meta.parse.

Something funny happens when I try it on your example:

julia> module M
          using ..Main
          +(a,b) = a^b
          sx = @sexpr (call, `+`, 2, 3)
       end
julia> M.sx
:(($(Expr(:incomplete, Base.Meta.ParseError("ParseError:\n# Error @ none:1:9\nMain.M.+\n#       └ ── premature end of input", Base.JuliaSyntax.ParseError(Base.JuliaSyntax.SourceFile("Main.M.+", 0, "none", 1, [1, 9]), Base.JuliaSyntax.Diagnostic[Base.JuliaSyntax.Diagnostic(9, 8, :error, "premature end of input")], :other)))))(2, 3))

As it turns out, Main.M.+ isn’t valid, it has to be spelled Main.M.:+.

julia> repr(M.:+)
"Main.M.+"

I would argue that this is a bug in repr. The manual says “As a rule of thumb, the single-line show method should print a valid Julia expression for creating the shown object.”, and I’ve found this to apply to built-in show (and therefore repr) in general.

Although if there’s another method I should be using, by all means, let me know.

There must be, right? This works:

julia> module M
          using ..Main
          +(a,b) = a^b
          sx = :($(+)(2, 3))
       end
WARNING: replacing module M.
Main.M

julia> M.sx
:((Main.M.+)(2, 3))

This is what I mean about not wanting to package this beast up :sweat_smile: Julia is a big language!

31

I realized that I was doing an unnecessary traversal through string and back to Expr, so it works correctly now:

julia> module M
          using ..Main
          +(a,b) = a^b
          sx = @sexpr (call, `+`, 2, 3)
       end
Main.M

julia> M.sx
:((Main.M.+)(2, 3))

julia> eval(ans)
8

This is respectable enough to show, I think. There’s no avoiding the parse/eval thing if starting with a string. Still kinda ugly, and there might be a better way to do it, idk.

 elseif expr.head == :macrocall &&
        expr.args[1] isa GlobalRef &&
        expr.args[1].name == Symbol("@cmd")
        return :( $(Base.eval(mod, Meta.parse(expr.args[3]))) )

But who know’s what’ll break next. I don’t like that this relies on expr.args[2] being a LineNumberNode, that seems fragile.

I have my answer to this and learned something in the process:

julia> dump(M.sx)
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: + (function of type typeof(Main.M.:+))
    2: Int64 2
    3: Int64 3

julia> dump(@sexpr (:call, +, 1, 2))
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol +
    2: Int64 1
    3: Int64 2

So the first argument to a :call Expr can be a function or a symbol. That’s cool, I had thought until now that Exprs were more AST-like than this.

I might still argue that repr should display the sort of function which the parser makes you quote as Base.:+, not Base.+, though calling it a bug is questionable. When I was adding some operator overloads it took some forum trawling for me to figure out that syntax.

32

How do you know that the symbol there is meant to be a global reference? I.e. what if I do

julia> let (+) = (-)
           Expr(:call, +, :a, :b)
       end
:((-)(a, b))

Besides, trying to auto-unquote things will doubtless be annoying if you want to include expressions and symbols in the quoted code, i.e.

julia> Expr(:call, :show, :(:a))
:(show(:a))

is not the same as

julia> Expr(:call, :show, :a)
:(show(:a))

and it also doesn’t help you with heads like if, function, struct, etc.

33

I don’t know that it’s meant to be a global reference, all I know is that in the syntax tree it’s a GlobalRef.

That said, I bet you found the next bug, lemme find out.

julia> let (+) = (-)
                  @sexpr (:call, `+`, a, b)
              end
:((+)(a, b))

julia> dump(ans)
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: + (function of type typeof(+))
    2: Symbol a
    3: Symbol b

Welp, that ain’t ideal.

So I know what I’d do in Lua: look for it in the upvalues. But I don’t know how to solve this problem in Julia. I seem to recall it being somewhat difficult to determine if a symbol is defined in the locals, is it possible at all?

34

This part I’ve got covered:

julia> @sexpr (call, show, :a)
:(show(:a))

julia> @sexpr (call, show, a)
:(show(a))

That part too:

julia> ABC
UndefVarError: `ABC` not defined

julia> @sexpr (:struct, false, (:<:, ABC, Pattern), (block, b))
:(struct ABC <: Pattern
      b
  end)

julia> eval(ans)

julia> ABC(1)
ABC(1)

julia> @sexpr (:struct, false, (:<:, DEF, Pattern))
:($(Expr(:struct, false, :(DEF <: Pattern))))

julia> cons(ans, @sexpr (block, b))
:(struct DEF <: Pattern
      b
  end)

julia> eval(ans)

julia> DEF(1)
DEF(1)

It’s special-cased to un-QuoteNode the head of a tuple, this whole sidequest started with wanting to build partial syntax using something other than Expr.

So the unescaping syntax doesn’t work for various reasons, but interpolation does the job:

julia> let pull! = push!
                  @sexpr (:call, $(pull!), a, b)
              end
:((push!)(a, b))

julia> module M
           using ..Main
           let + = (a,b) -> a^b
               global sx = @sexpr (call, $(+), 2, 3)
           end
       end
Main.M

julia> M.sx
:((Main.M.var"#1#2"())(2, 3))

julia> eval(M.sx)
8

So I wonder if maybe just respecting the Julia syntax for unescaping, without trying to get fancy, is the better way to go.

In fact I wonder why I thought I needed it. One of those first-cup-of-coffee ideas, those don’t always pan out.

35

In other news, I’ve arrived at an acceptable cdr function, given that Julia Exprs are not, in fact, composed of cons cells.

julia> ifdef = @sexpr (:if, (call, <, a, b), (call, f, a, b))
:(if a < b
      f(a, b)
  end)

julia> elseifdef = @sexpr (:elseif, (call, isa, a, Symbol), (call, g, a, b, :c))
:(elseif a isa Symbol
      g(a, b, :c)
  end)

julia> cons(ifdef, elseifdef)
:(if a < b
      f(a, b)
  elseif a isa Symbol
      g(a, b, :c)
  end)

julia> cdr(ans)
:(elseif a isa Symbol
      g(a, b, :c)
  end)

julia> @sexpr (call, +, 1, 2)
:(1 + 2)

julia> cdr(ans)
NotLispError("can't fake cdr unless expr.args[end] is an Expr")

It could just return expr.args[end], but in any useful case that would just propagate the error, you really want it to be an Expensive Cons Cell from Luxury Lisp.

It’s even less like bona-fide cdr than the first one though, but it does replace its function in the macro-building context, sorta. When you’re consing up an s-expression, you usually have the tail pair available to pass the cdr to the next tail-recursive operation. … I’m not going to say it, but I’m thinking it very hard

36

So I removed the special handling of quoted :(...) syntax because I belatedly realized that @sexpr is a better hybrid syntax than I thought:

julia> y = 27
27

julia> @sexpr (:if, (2 < 3), f(x,$y,z))
:(if 2 < 3
      f(x, 27, z)
  end)

julia> @sexpr (:if, (2 < $y && x isa Expr))
:($(Expr(:if, :(2 < 27 && x isa Expr))))

julia> cons(ans, :(return f(x)))
:(if 2 < 27 && x isa Expr
      return f(x)
  end)

That’s pretty nice! It’s more like :( ) but you can use an Expr-like syntax to write partial expressions.

The only wrinkle is that you have to spell ~y as (:call, ~, y), because ~ is reserved for QuoteNode syntax. I have to admit, I haven’t figured out what sort of “advanced metaprogramming tasks” QuoteNode is good for, so I can hardly venture a guess as to whether reserving a character for it is useful or eloquent. I’ve only ever seen them in the context of symbols, and this syntax supports those already.

The behavior is also not identical to The Book, compare:

julia> @sexpr (:if, (y == ~(fn(a, $b, c))))
:($(Expr(:if, :(y == $(QuoteNode(:($(Expr(:quote, :(fn(a, $(Expr(:$, :b)), c)))))))))))

julia> @sexpr (~($(2 + 3)))
:($(QuoteNode(:($(Expr(:quote, :($(Expr(:$, :(2 + 3))))))))))

julia> eval(ans)
:($(Expr(:quote, :($(Expr(:$, :(2 + 3)))))))

julia> eval(ans)
5

To the example in the manual:

julia> eval(QuoteNode(Expr(:$, :(1+2))))
:($(Expr(:$, :(1 + 2))))

julia> eval(ans)
syntax: "$" expression outside quote

Three things going on here: I couldn’t figure out how to get ~ to behave like QuoteNode, I don’t know what QuoteNode is good for, and I expected it to do what ~ does now! If I don’t stick in the extra level of quoting, the $(1 + 2) just gets evaluated to 5, then wrapped in a QuoteNode. I’d appreciate suggestions on how to prevent this, if anyone is still following along on my adventures.

If I make the QuoteNode like this:

Expr(:$, :($(QuoteNode(toquote))))

I get similar, but still not identical, behavior

julia> @sexpr (~($(2 + 3)))
:($(Expr(:$, :(2 + 3))))

julia> eval(ans)
syntax: "$" expression outside quote

julia> QuoteNode(Expr(:$, :(2 + 3)))
:($(QuoteNode(:($(Expr(:$, :(2 + 3)))))))

julia> eval(ans)
:($(Expr(:$, :(2 + 3))))

julia> eval(ans)
syntax: "$" expression outside quote

The escaping process seems to be un-quotenoding my QuoteNode.

I’ve obviously fallen down a deep rabbit hole here, and I think I’m going to defer the question of what ~ should mean until I find a need for a non-symbol use of QuoteNode. The answer might turn out to be removing it from the macro entirely.

37

So if I double-wrap the QuoteNode like this, one gets through the escaping process:

return QuoteNode(Expr(:$, QuoteNode(toquote)))

That gives the expected (?) behavior:

julia> @sexpr (:if, (2 < 3), $(QuoteNode(Expr(:call, :f, :x, Expr(:$, :y), :z))))
:(if 2 < 3
      $(QuoteNode(:(f(x, $(Expr(:$, :y)), z))))
  end)

julia> cons(ans, :(f(z,c,z)))
:(if 2 < 3
      $(QuoteNode(:(f(x, $(Expr(:$, :y)), z))))
  else
      f(z, c, z)
  end)

julia> eval(ans)
:(f(x, $(Expr(:$, :y)), z))

julia> eval(ans)
syntax: "$" expression outside quote

julia> @sexpr (:if, (2 < 3), ~(f(x,$y,z)))
:(if 2 < 3
      $(QuoteNode(:(f(x, $(Expr(:$, :y)), z))))
  end)

julia> cons(ans, :(f(z,c,z)))
:(if 2 < 3
      $(QuoteNode(:(f(x, $(Expr(:$, :y)), z))))
  else
      f(z, c, z)
  end)

julia> eval(ans)
:(f(x, $(Expr(:$, :y)), z))

julia> eval(ans)
syntax: "$" expression outside quote

Now ~ acts like the manual’s example:

julia> @sexpr (~($(1+2)))
:($(QuoteNode(:($(Expr(:$, :(1 + 2)))))))

julia> eval(eval(ans))
syntax: "$" expression outside quote

julia> QuoteNode(Expr(:$, :(1+2)))
:($(QuoteNode(:($(Expr(:$, :(1 + 2)))))))

julia> eval(eval(ans))
syntax: "$" expression outside quote

Welp. I’m still not sure what QuoteNode is good for, but nevertheless, the macro now produces them. If I ever come up with an application for this, I’ll let y’all know right away.