How to evaluate symbol as a "variable" inside a macro?

General Usage
, , ,
1

Hello, I’m currently trying to implement merge on ComponentArrays. I’ve taken inspiration from NamedTuples in the stdlib here, and wrote a macro and a function: (instead of the Expr)

using ComponentArrays
c1 = ComponentArray(a=2, b=5)
c2 = ComponentArray(b=1, g=2)

macro append_component_array(parent_array, keyname, val)
    return esc(:($parent_array = ComponentArray($parent_array, $keyname = $val) ))
end

function merge_test(cvec1::ComponentVector{T1}, cvec2::ComponentVector{T2}) where {T1, T2}
    typed_dict = ComponentVector{promote_type(T1, T2)}(cvec1)
    for key in valkeys(cvec2)
        keyname = ComponentArrays.getval(key)
        val = cvec2[key]
        @append_component_array(typed_dict, keyname, val)
    end
    typed_dict
end

However, I can’t seem to get keyname to evaluate properly in the macro to a “variable”:

>julia merge_test(c1,c2)
ComponentVector{Int64}(a = 2, b = 5, keyname = 2)

I can do this with eval easily, though it will kill precompilation.

2

Macros don’t evaluate symbols, it expands during the definition, before the method is ran and before the variables have values. Your macro just interpolates :typed_dict, :keyname, and :val into a line typed_dict = ComponentArray(typed_dict, keyname = val). Forgetting the macro for a second, you need a different line where a keyword is a runtime Symbol value rather than a fixed variable name. Try typed_dict = ComponentArray(typed_dict; keyname => val)

3

Huh, that works.

Though why does dump not work?

4

I don’t know what you mean, you didn’t use dump anywhere before.

5

I tried to use dump to use keyname as a variable, thinking it would work based on how I understood it. Though, the result is either a crash (if I try to interpolate the dump), or nothing (if I don’t).

6

Provide the dump call code, I can’t imagine what you mean.

7 
macro append_component_array(parent_array, keyname, val)
    return esc(:($parent_array = ComponentArray($parent_array, $dump(:(:keyname)) = $val) ))
end

or

macro append_component_array(parent_array, keyname, val)
    dump(:(:keyname))
    return esc(:($parent_array = ComponentArray($parent_array, $keyname = $val) ))
end

or

macro append_component_array(parent_array, keyname, val)
    key = dump(:(:keyname))
    return esc(:($parent_array = ComponentArray($parent_array, $key = $val) ))
end
8

dump prints a value in a tree-like way to a maximum depth defaulting at 8, then returns nothing. You’re interpolating the value nothing where a Symbol instance is expected for a keyword.

1 Like
9

As the question stands, I doubt meta programming is the answer. In particular, I don’t see the value of the macro over a function.

Maybe you could explain what the end goal is and why you believe meta programming is the correct way to get there. I’m saying this, because more often than not, it isn’t.

1 Like
10

I didn’t see a way to implement it via a function, but apparently there is.

11

I am still not really sure what “it” is. If you simply want to merge two ComponentArrays, the package has you covered already:

c1 = ComponentArray(a=2, b=5, c=[10, 20])
c2 = ComponentArray(b=1, g=2)

julia> ComponentArray(c1; c2...)
ComponentVector{Int64}(a = 2, b = 1, c = [10, 20], g = 2)
12

I’ve tried using it, though it doesn’t support Reverse AD. I thought if I try to implement it like Base.merge for NamedTuple, maybe it’ll work.

13

I see. The better course of action is to open an issue with ComponentArrays.jl, or write a ChainRulesCore.rrule yourself. Ideally ultimately making a PR.

The issue however has already been raised Missing ChainRule for `ComponentVector(a; b...)` · Issue #207 · jonniedie/ComponentArrays.jl · GitHub and it looks like a solution might be near :slight_smile:

I would imagine writing a proper pullback will turn out to be a lot more fruitful than implementing merge again.

1 Like