Expressions with Arbitrary Objects in their `args` Arguments

Internals & Design
, ,
1

I checked Core.Expr, Core.eval, Metaprogramming, and Julia ASTs, and I wasn’t able to find an answer. Is this behavior intended and supported?

Let’s build a couple of expressions:

julia> ax1, ax2 = :([1:5;]), [1:5;]
(:([1:5;]), [1, 2, 3, 4, 5])

julia> ex1, ex2 = :(1 .+ $ax1), :(1 .+ $ax2)
(:(1 .+ [1:5;]), :(1 .+ [1, 2, 3, 4, 5]))

Notice that while ex1 has had an expression interpolated into it, ex2 has had an instance of Vector{Int64} interpolated into it. To confirm:

julia> dump(ex2)
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol .+
    2: Int64 1
    3: Array{Int64}((5,)) [1, 2, 3, 4, 5]

julia> typeof(ex2.args[3])
Vector{Int64} (alias for Array{Int64, 1})

Even though ex2.args[3] isn’t an expression, symbol, string, or numeric literal, the expression “just works!™”:

julia> eval(ex1), eval(ex2)
([2, 3, 4, 5, 6], [2, 3, 4, 5, 6])

It also works if you make any other sort of object (mutable or immutable, including user-defined objects) an expression args argument.

On the upside, this behavior allows some nice efficiencies—not having to destruct and restruct an object, so eval can run faster and the code can sometimes be simpler. On the downside, if you’re not careful, a mutable collection can get accidentally mutated. Example:

julia> @generated foo(q) = let x = [2]; :($x, q ? $x.^2 : $x, $x.^3) end
       a, b = foo(true), foo(false)
(([2], [4], [8]), ([2], [2], [8]))

julia> a[1][1] = 9
       a, b
(([9], [4], [8]), ([9], [9], [8]))

Not a big deal, but it could be worth documenting.

So my question is thus:

Is this behavior intended and supported?

I ask because I’m pretty sure I’ve inadvertently interpolated objects like this, and if it’s intended then I want to leverage it when I can…

2

I believe @fonsp exploited this ability to have partially evaluated expressions in a demo Pluto notebook, allowing you to “step through time” and see how expressions evaluate from the inside-out.

Forgive me if I remember incorrectly, but I can’t find the notebook now. Any pointers, @fonsp? That was a very cool idea indeed!

3

Metaprogramming noob here… but the docs you linked say at some point

Expressions provided by the parser generally only have symbols, other expressions, and literal values as their args, whereas expressions constructed by Julia code can have arbitrary run-time values without literal forms as args.

And in an example a bit below that it is stated

julia> a = 1;

julia> ex = Expr(:call, :+, a, :b)
:(1 + b)

julia> a = 0; b = 2;

julia> eval(ex)
3
  • The value of the variable a at expression construction time is used as an immediate value in the expression. Thus, the value of a when the expression is evaluated no longer matters: the value in the expression is already 1, independent of whatever the value of a might be.

Sounds like it’s really the value of the variable that will be used in an Expr, whatever it might be. Is this the behavior that you are referring to, or did I miss something?

5 Likes
4

Yes

1 Like
5

Ah, that was the key phrase I had missed! Thanks for the spot!

(There’s an example of interpolating a Tuple, which in retrospect should have been sufficient for me, but for some reason my mind dismissed its relevance due to it being isbits.)

3 Likes
6

An example of this in Base Julia is the regex string macro, as you can see here it directly puts the Regex object into the AST:

4 Likes
7

This is in PlutoTest.jl :blush:

2 Likes