Eval scoping in macros, or removing eval completely

General Usage
1

Hi all,

Took a stroll into marco coding and got totally lost. Trying to solve this issue.

Breakdown:
We have a simple enough macro helper @agent, which generates a few different types of struct depending on input.

@agent Zombie OSMAgent begin
  infected::Bool
end
# --->
mutable struct Zombie <: AbstractAgent
      id::Int64
      pos::Tuple{Int64,Int64,Float64}
      route::Array{Int64,1}
      destination::Tuple{Int64,Int64,Float64}
      infected::Bool
end

The current implementation looks like:

macro agent(name, base, fields)
    base_type = Core.eval(@__MODULE__, base)
    base_fieldnames = fieldnames(base_type)
    base_types = [t for t in base_type.types]
    base_fields = [:($f::$T) for (f, T) in zip(base_fieldnames, base_types)]
    res = :(mutable struct $(esc(name)) <: AbstractAgent end)
    push!(res.args[end].args, base_fields...)
    push!(res.args[end].args, map(esc,fields.args)...)
    return res
end

I want to get around using the eval call, since it’s breaking incremental compilation if this macro is called within a module (see the issue raised above). This also happens if I use the global eval rather than the module scoped one.

Is there a way I can get the fieldnames and .types from my base variable/expression without having to call eval? If not, is there a way to satisfy the compiler in this instance?

2

You could create the OSMAgent type with a macro, which at the same time creates a @agent_OSMAgent macro which can then contain all the info it needs as it is constructed with OSMAgent. I use something similar in Parameters.jl, in which the @with_kw macro constructs a type devinition and also a macro @unpack_ThatType which unpacks all fields into variables: Parameters manual · Parameters.jl. Unfortunately, the code of Parameters.jl is a bit messy, so not sure whether you could actually use it, but you can try.

3

You can use getfield on the module to look up a symbol in a module. If the representation of the type is unqualified and non-parametric this is all you need.

An example of a qualified representation is Base.RefValue. Since it is qualified you will need to run getfield on the qualifiers recursively until you get to the type itself. In this case you would need to use getfield to extract the field by :Base in the current module and then check the :RefValue within it. This is required as getfield only accepts symbols as arguments and the path to the type is an expression.

3 Likes
4

Thanks @WschW, that’s got me on the right track. I’m happy to just support unqualified types, but will need to cover a specific parametric type.

GridAgent{Int} and ContinuousAgent{Int}. I’ll take a look at that myself soon, but if you know a nice way to build the type signatures, that’d be really helpful.

Current (not working) progress:

    if hasfield(typeof(base), :args)    
        # This option assumes only one parametric    
        # type for dimensionality. Will need to be    
        # modified for more complex types.    
        base_type = getfield(@__MODULE__, base.args[1])    
        base_types = collect(base_type.body.types) #<-- These are not currently appropriate. Need to incorporate args[2]
    else                                                                                                                               
        base_type = getfield(@__MODULE__, base)                                                                                        
        base_types = collect(base_type.types)                                                                                          
    end                                                                                                                                
    base_fieldnames = fieldnames(base_type)
5

That’s a nice approach. I’m kind of interested in this, having wanted something similar myself, mainly a way to splice fields from one struct into another struct, as a kind of inheritance. Similar to what Classes.jl does, but without some of the other stuff, and without needing the “superclass” to be defined in any special way. Anyway, sorry to slightly hijack this thread, but here’s a MWE using your idea:

julia> using MacroTools

julia> macro extends(structdef)
           fields = []
           for f in structdef.args[3].args
               isexpr(f, :(...)) ? append!(fields, fieldnames(getfield(__module__, f.args[1]))) : push!(fields, f)
           end
           structdef.args[3] = :(begin $(fields...) end)
           structdef
       end
@extends (macro with 1 method)

julia> struct Foo
           x
           y
       end

# I think this would be a nice straightforward syntax for it:
julia> @extends struct Bar
           Foo...
           z
       end

julia> Bar |> fieldnames
(:x, :y, :z)

Its also straightforward how to make parametric types work too as long as the types were explicit, like

@extends struct Foo
    Complex{Float64}...
    other
end

would be easy since all the info we need is here:

julia> fieldnames(Complex{Float64})
(:re, :im)

julia> fieldtypes(Complex{Float64})
(Float64, Float64)

I’m curious whether there’s a way to make it work with type arguments too, like could you do this?:

@extends struct Foo{T}
    Complex{T}...
    other
end

The necessary info must be somewhere but I can’t find a Julia function that exposes it. Maybe with a C API call?

2 Likes
6

If you want to support qualified types something like this function:

can be used to split the qualified type into an array of the qualifying symbols and the name of the type which can help in looking up the type.

Side note: This package uses eval for the same thing you have been using eval for and while I have been meaning to get rid of it and a few other things I never got around to it as no one including myself uses it.

Edit: Also for this kind of use case you may want to look into this package:

2 Likes
7

Thanks all, these are really helpful! I’ll dig a little deeper and report back once I have something workable.

8

Ok, think I got it working. No C API needed, it always surprises me what you can do in pure Julia. I called it StructExtender.jl for now, example from its readme:

struct Foo{X,Y}
    x :: X
    y :: Y
end

@extends struct Bar{X,Y,Z}
    Foo{X,Y}...
    z :: Z
end

# equivalent to defining:
struct Bar{X,Y,Z}
    x :: X
    y :: Y
    z :: Z
end

Feel free to play with it, I think it could be used to solve @Libbum’s original problem. Will probably sit on it for a bit then consider registering. Curious if any of you guys have ideas for the name of the package and macro, maybe there’s something better / clearer.

4 Likes