Using `eval` inside `quote`

General Usage
1

I’ve recently added a new feature to CounterfactualExplanations.jl that adds support for multi-processing through MPI. On the user-end, this is handled through a simple macro:

ces = @with_parallelizer parallelizer begin
    generate_counterfactual(
        xs,
        target,
        counterfactual_data,
        M,
        generator
    )
end

Now I realised that this causes an issue for keyword arguments that are passed to generate_counterfactual as Symbols, because they are turned into QuoteNodes inside the macro, e.g.

ces = @with_parallelizer MPIParallelizer() begin
    generate_counterfactual(
        xs,
        target,
        counterfactual_data,
        M,
        generator;
        initialization=:identity
    )
end

throws an error:

╭──────────────────────────────────────────────────────────────── TypeError ───────────────────────────────────────────────────────────────╮
│                                                                                                                                          │
│  TypeError: in keyword argument initialization, expected Symbol, got a value of type QuoteNode                                           │
│                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

In this PR I’ve added a fix that relies on call eval on the QuoteNode inside the final quote that is returned by the macro:

macro with_parallelizer(parallelizer, expr)
    
    [...]

    # Parallelize:
    output = quote
        @assert CounterfactualExplanations.parallelizable($f) "`f` is not a parallelizable process."
        kws = [Pair(k, typeof(v) == QuoteNode ? eval(v) : v) for (k, v) in $aakws]
        output = CounterfactualExplanations.parallelize($pllr, $f, $escaped_args...; kws...)
        output
    end
    return output
end

This feels dodgy (as do most things when I dare to write a macro) but if I understand the following section of the docs on macro hygiene correctly, it is fine/standard?

[…] For example, many macros simply wrap their arguments in a QuoteNode or other similar Expr. Some examples of this include @task body which simply returns schedule(Task(() -> $body)) , and @eval expr , which simply returns eval(QuoteNode(expr)) .

Is this indeed fine? If not, any suggestions as to how I can deal with the QuoteNode or improve/avoid the macro altogether?

Thanks!

2

In general no - the only reason @eval has an eval is because that’s what it does (this is not circular logic - @eval is quite literally an eval). Macros generally shouldn’t need to eval.

What you’d really want to do is create the entire expression for that final call:

CounterfactualExplanations.parallelize($pllr, $f, $escaped_args...; kws...)

and just give it the keywords part of the initial expression. You can get only those keywords already by checking the second argument of the call expression:

julia> Meta.@dump f(a,b; c=:d)
Expr
  head: Symbol call
  args: Array{Any}((4,))
    1: Symbol f
    2: Expr
      head: Symbol parameters
      args: Array{Any}((1,))
        1: Expr
          head: Symbol kw
          args: Array{Any}((2,))
            1: Symbol c
            2: QuoteNode
              value: Symbol d
    3: Symbol a
    4: Symbol b

A bit of an orthogonal note - @assert is not guaranteed to run at all optimization levels. See also its docstring. So checks like here or here really ought to use a proper conditional & throw.

2 Likes
3

Thanks very much @Sukera for the quick response. I’m not sure this exactly what you had in mind but your MWE has given me an idea how to avoid eval by just retrieving the value from the QuoteNode before defining the quote (see here). Still feels somewhat hacky but hopefully that’s a cleaner/safer approach.

I’ve also fixed the @assert, thanks for flagging.

Thanks!

1 Like