Generate an array in a macro

OvidiusCicero · April 6, 2019, 12:34pm

I have some variables, eg

a=1;b=2;c=3;

What I want is to generate an array (or tuple) of the following:

[("a", 1),
 ("b", 2),
 ("c", 3)]

So far I got the conversion for one variable

macro convertarg(var)
    return :(($(string(var))), $(esc(var)))
end

julia> @convertarg(a)
("a", 1)

My first idea was to call convertarg in a loop from within another macro:

macro convertargs(vars...)
	pairs = []
	for var in vars
		append!(pairs, (@convertarg(var),))
	end
	return pairs
end

which does not work since it variables must be quoted also in convertargs. Also directly putting the code of convertarg at the position of the macro call does not work for the same reason. When quoting the whole expression I cannot access the macro arguments.

How can I obtain in array as wanted? I cannot wrap around that the expressions within the array must be quoted but not the returned array itself.

I am also thankfull for some explanations since I seemingly misunderstand something in how macros work.

sagartewari01 · April 6, 2019, 1:19pm

This macro should do what you want:

macro cas(args...)
    ret = Expr(:vect)
    for a in args
        push!(ret.args, :(($(string(a)), $(esc(a)))))
    end
    return ret
end

The head of an expression tells us the type of expression, and the args describe the expression. The first line creates an expression with symbol vect as first element, and the for loop adds the ast representations of the tuples to the args of the vector, which define the contents of a vector.

OvidiusCicero · April 6, 2019, 1:30pm

Thank you!

I tried out the followingto get out a tuple from the code:

macro cas(args...)
    ret = Expr(:tuple)
    for a in args
        push!(ret.args, :(($(string(a)), $(esc(a)))))
    end
    return ret
end

and it works. So :vect for Arrays and :tuple for tuples. Can this be generalized? Or are those things documented?

rdeits · April 6, 2019, 1:31pm

Why do you want to use a macro for this? It would be simpler and just as efficient to use a function to construct the result you’re asking for.

OvidiusCicero · April 6, 2019, 1:32pm

afaik it is not possible to obtain the name of a variable as string in a function

rdeits · April 6, 2019, 1:34pm

Oh, my mistake, I didn’t notice that you wanted the names of the variables, not their values. Carry on

sagartewari01 · April 6, 2019, 1:36pm

Try dump(:(1+1 == 2))

OvidiusCicero · April 6, 2019, 2:50pm

thank you again!

hhaensel · September 3, 2020, 8:23am

I had a similar problem yesterday and your post helped me solving it

I digged a bit further and found that the following code is simpler and even more efficient:

macro cas2(args...)
    s = String.(args)
    v = eval.(args)
    return Tuple(zip(s, v))
end

or if you are a friend of one-liners you can compress it to

macro cas3(args...)
    return Tuple(zip(String.(args), eval.(args)))
end

Benchmarking shows

julia> @btime @cas(a, b, c);
  330.636 ns (4 allocations: 160 bytes)

julia> @btime @cas2(a, b, c);
  2.552 ns (0 allocations: 0 bytes)

julia> @btime @cas3(a, b, c);
  2.552 ns (0 allocations: 0 bytes)

simeonschaub · September 3, 2020, 8:59am

You are not measuring what you think you are here, because macro expansion happens before the timing runs. There is never a good reason to use eval in a macro, since you will run into all kinds of scoping and world age issues. eval is also really slow, you are just not measuring it here, because the call happens at macro espansion time.

hhaensel · September 3, 2020, 11:42am

Thanks for pointing to that!
Shall I delete it? Or we leave it as a bad example…

hhaensel · September 4, 2020, 7:41pm

This macro programming can be a bit brain twisting (I must confess that I am a total beginner of macro writing)

For my own understanding I tried different versions to achieve the same result. As I feel that this part of julia is not so well documented in the standard docs, I put the versions here and would be happy to receive comments on what is good Julian style.

# as proposed by @sagartewari01
macro cas(args...)
    ret = Expr(:tuple)
    for a in args
        push!(ret.args, :(($(string(a)), $(esc(a)))))
    end
    return ret
end

# the shortest version, could even be made one-liner ...
macro cas2(args...)
    aa = (:(($(string(a)), $(esc(a)))) for a in args)
    return Expr(:tuple, aa...)
end

# similar to @cas, but using a comprehension
macro cas3(args...)
    aa = Expr(:tuple)
    aa.args = [:(($(string(a)), $(esc(a)))) for a in args]
    return aa
end

# along the lines of classical function programming
macro cas4(args...)
    s = string.(args)
    v = Expr(:tuple, esc.(args)...)
    return :(Expr(:tuple, (zip($s, $v))...))
end

I have also listed comparative timings, @simeonschaub could perhaps comment, how good that reflects the true performance (or how it could be done better )

julia> @btime @cas(a1,a2, a2, a3, a4, a5);
  641.316 ns (7 allocations: 304 bytes)

julia> @btime @cas2(a1,a2, a2, a3, a4, a5);
  642.318 ns (7 allocations: 304 bytes)

julia> @btime @cas3(a1,a2, a2, a3, a4, a5);
  644.392 ns (7 allocations: 304 bytes)

julia> @btime @cas4(a1,a2, a2, a3, a4, a5);
  3.555 μs (22 allocations: 1008 bytes)

ffevotte · September 7, 2020, 9:11am

I think the one central idea one should keep in mind when writing macros is this:

Macros are not magic: they are a way to transform input syntax into a desired output syntax

In particular, this means:

when writing a function, you first think about which results it should compute given a set of input arguments. Likewise, when writing a macro, the first step should be to think about what code the macro should produce when given a specific input. In this case, even before starting to write the macro, we decided that
```
@cas(a, b, c)
```
should produce this specific julia code
```
(("a", a), ("b", b), ("c", c))
```
We took that decision because this particular expression is better (in terms of runtime performance) than others (e.g. using a Vector of Tuples). But this is the only stage where runtime is even a consideration (and, again, it happens before we even start writing the macro)
Now that we know what code the macro should produce, we don’t care about runtime performance. Either a given implementation generates the code we want (and it will achieve the performance associated to that code), or it does not (incorrect => performance does not matter). So we should be primarily concerned with the correctness of a macro, i.e. check that the macro does indeed produce the desired code, when given the right input. The tool to do this is @macroexpand; a working implementation of the macro should behave like this:
```
julia> @macroexpand @cas(a,b, c)
:((("a", a), ("b", b), ("c", c)))
```
Again, if we manage to get several correct implementations of the macro, they should all produce the same code (by definition of what “correct” means for a macro) and therefore should always have the same runtime speed.
However, different implementations of the macro might have different expansion-time performances. This will affect the compile-time performance of your code, and if you care about such issues, you can benchmark the macro expansion phase itself. One way of doing this (there might be others) is to turn the macro into a function, and benchmark the function itself as usual:
```
julia> function cas_fun(args...)
           ret = Expr(:tuple)
           for a in args
               push!(ret.args, :(($(string(a)), $(esc(a)))))
           end
           return ret
       end
cas_fun (generic function with 1 method)

julia> @btime cas_fun(:a, :b, :c)
  593.309 ns (24 allocations: 1.38 KiB)
:((("a", $(Expr(:escape, :a))), ("b", $(Expr(:escape, :b))), ("c", $(Expr(:escape, :c)))))
```
(note how the function call had to be adapted)

Applying such a method here:

as early as step 2, we notice that @cas4 is incorrect from our code generation perspective; no wonder it does not yield the same performance as other implementations
likewise, as early as step 2 we know that @cas, @cas2 and @cas3 will feature the same runtime performance => no need to test again (such benchmarks should already have been done at step 1, when trying to determine the best expression for the macro to produce)
if we benchmark the macro expansion time of all versions, it looks like the original @cas implementation is the fastest:

function cas_fun(args...)
    ret = Expr(:tuple)
    for a in args
        push!(ret.args, :(($(string(a)), $(esc(a)))))
    end
    return ret
end

function cas2_fun(args...)
    aa = (:(($(string(a)), $(esc(a)))) for a in args)
    return Expr(:tuple, aa...)
end

function cas3_fun(args...)
    aa = Expr(:tuple)
    aa.args = [:(($(string(a)), $(esc(a)))) for a in args]
    return aa
end

julia> using BenchmarkTools

julia> @btime cas_fun(:a, :b, :c)
  587.229 ns (24 allocations: 1.38 KiB)
:((("a", $(Expr(:escape, :a))), ("b", $(Expr(:escape, :b))), ("c", $(Expr(:escape, :c)))))

julia> @btime cas2_fun(:a, :b, :c)
  2.735 μs (27 allocations: 1.48 KiB)
:((("a", $(Expr(:escape, :a))), ("b", $(Expr(:escape, :b))), ("c", $(Expr(:escape, :c)))))

julia> @btime cas3_fun(:a, :b, :c)
  892.550 ns (27 allocations: 1.61 KiB)
:((("a", $(Expr(:escape, :a))), ("b", $(Expr(:escape, :b))), ("c", $(Expr(:escape, :c)))))

hhaensel · September 8, 2020, 7:46am

Wow, that is a very well written answer and provides lots of clarity!
This text could be part of the regular docs.
Thank you so much!

Topic		Replies	Views
Can I concatenate expressions in macro? General Usage	5	1726	October 5, 2021
Help writing macro which returns an eval expression General Usage	7	640	November 13, 2019
I need help with my Macros (newbie question) General Usage	2	270	December 3, 2021
Macro to create variables from arrays of names and values General Usage metaprogramming	14	2917	January 15, 2021
How to produce the code "f(a...,b...)" General Usage	5	622	March 29, 2017

Generate an array in a macro

Related topics