Code Generation with macro

[Andy_Zhang]

Background

I am writing a package that provides some structures of matrices, and I want them to have different properties.

module MyModuleA

export
    TypeA,
    properties

# function interfaces
properties(::Type{<:AbstractMatrix})::Vector{Symbol} = []
properties(m::AbstractMatrix{}) = properties(typeof(m).name.wrapper)

# struct
struct TypeA{T<:Int} <: AbstractMatrix{T}
    n::Int
end

properties(::Type{TypeA}) = [:symmetric]
end

I would like the user to add new types and set properties for it:

module UserModuleA

using ..MyModuleA
import ..MyModuleA: properties

export
    TypeB

struct TypeB{T<:Int} <: AbstractMatrix{T}
    n::Int
end

properties(::Type{TypeB}) = [:symmetric, :inverse]
end

julia> using .MyModuleA, .UserModuleA

julia> properties(TypeB)
2-element Vector{Symbol}:
 :symmetric
 :inverse

Question

The general approach has two important steps to set properties for my new type:

import ..MyModuleA: properties
properties(::Type{TypeB}) = [:symmetric, :inverse]

Now we want some new features:

1. add a constant vector in MyModuleA saving all valid properties
2. for convenience, avoid user to explicitly import the function
3. (addtional) define a type Property packing these properties symbols

   struct Property
       name::Symbol
   end
   
   # then it comes like that which is not a good design
   properties(::Type{TypeB}) = [Property(:symmetric), Property(:inverse)]
   

I would like to design a macro to solve these problems. The new code should looks like that:

module MyModuleA

export
    TypeA,
    @properties,
    properties

const PROPERTIES = [:symmetric, :inverse]

# properties macro
macro properties(type::Symbol, ex::Expr)
    return quote
        properties(::Type{$(esc(type))}) = [Property(property) for property = $esc(ex)]
    end
end

# function interfaces
properties(::Type{<:AbstractMatrix})::Vector{Symbol} = []
properties(m::AbstractMatrix{}) = properties(typeof(m).name.wrapper)

# struct
struct TypeA{T<:Int} <: AbstractMatrix{T}
    n::Int
end

properties(::Type{TypeA}) = [:symmetric]
end

module UserModuleB

using ..MyModuleA

export
    TypeB

struct TypeB{T<:Int} <: AbstractMatrix{T}
    n::Int
end

@properties TypeB [:symmetric, :inverse]
end

This macro

macro properties(type::Symbol, ex::Expr)
    return quote
        properties(::Type{$(esc(type))}) = [Property(property) for property = $esc(ex)]
    end
end

does not work, but the import statement is still not removed and it is not stable as the module name may be changed in the future, also it may be called in MyModuleA.

My Attempts

• esc the whole function definition
• @eval the function definition
• add import statement in the quote

All these attempts tries are failed. It seems like a Code Generation inside a macro. The type and list of symbols should be escaped, but the properties function should not be escaped.

[abraemer]

To avoid requiring the import/using of the properties function you could simply fully qualify it MyModuleA.properties. I am unsure where this should be a macro though. I think I would make this a function that @evals a new method. This makes it easier to check arguments.

Some comments on the design:
Instead of using Symbols make the traits types amd return a tuple from the properties function. This way every property or a check for a property can be inferred statically. Also it improves discoverability and removes the need for having some list of possible properties. The only difference is that the user needs to using these but I don’t think that that’s unreasonable.

So with this in mind, here’s my suggestion for your code:

module MyModuleA

export TypeA, register_properties_for_type, properties, SymmetricProperty, InverseProperty
# properties
# either give these rather long names such that they don't clash with other commonly used names (e.g. Symmetric is used by LinearAlgebra.jl)
# or put these into an exported submodule e.g. Properties
# and access using that like Properties.Symmetric
abstract type AbstractProperty end
struct SymmetricProperty <: AbstractProperty end
struct InverseProperty <: AbstractProperty end

properties(x) = properties(typeof(x)) # convenience dispatch so you can call it on instances
function register_properties_for_type(T::Type, properties::AbstractProperty...)
    @eval properties(::Type{$T}) = $properties    
end

struct TypeA{T<:Int} <: AbstractMatrix{T}
    n::Int
end

# use the function to define the properties
register_properties_for_type(TypeA, Symmetric())

end #module

module UserModuleA
using ..MyModuleA

export TypeB

struct TypeB{T<:Int} <: AbstractMatrix{T}
    n::Int
end

register_properties_for_type(TypeB, SymmetricProperty(), InverseProperty())

end #module

[Andy_Zhang]

Hi, thank you for your suggestions and idea to add types for properties, these are very helpful.

I still wondering if it is possible to add a macro interface to register properties? It is much more looks like a interface for metaprogramming.

[abraemer]

@Andy_Zhang

I still wondering if it is possible to add a macro interface to register properties

Sure you can do something but why? What do you want to achieve that justifies a macro?

A simple macro like this does not make much sense:

macro register_properties_for_type(type, props...)
    return :(MyModuleA.register_properties_for_type($type, $(props)...))
end

[Andy_Zhang]

@abraemer I really appreciate your idea on property types and how to write the registration function and macro.

Here is my final solution, in case anyone facing the same issue:

module MyModuleA

export
    TypeA,
    PropertyTypes,
    Property,
    list_properties,
    @properties,
    properties

module PropertyTypes
abstract type AbstractProperty end
struct Symmetric <: AbstractProperty end
struct Inverse <: AbstractProperty end
struct IllCond <: AbstractProperty end
struct PosDef <: AbstractProperty end
struct Eigen <: AbstractProperty end
struct Sparse <: AbstractProperty end
struct Random <: AbstractProperty end
struct RegProb <: AbstractProperty end
struct Graph <: AbstractProperty end
end

struct Property
    name::Symbol
end

const PROPERTIES = Dict{Type{<:PropertyTypes.AbstractProperty},Property}(
    type => Property(symbol) for (type, symbol) = Dict(
        PropertyTypes.Symmetric => :symmetric,
        PropertyTypes.Inverse => :inverse,
        PropertyTypes.IllCond => :illcond,
        PropertyTypes.PosDef => :posdef,
        PropertyTypes.Eigen => :eigen,
        PropertyTypes.Sparse => :sparse,
        PropertyTypes.Random => :random,
        PropertyTypes.RegProb => :regprob,
        PropertyTypes.Graph => :graph,
    )
)

# list properties
list_properties() = collect(values(PROPERTIES))

# check property types
function check_propertie_types(props::DataType...)
    for prop = props
        prop <: PropertyTypes.AbstractProperty || throw(ArgumentError("$prop is not a property type"))
    end
end

# check properties exists
function check_properties_exists(props::Property...)
    for prop = props
        prop ∈ values(PROPERTIES) || throw(ArgumentError("Property $prop not exists"))
    end
end

# properties types to properties
function property_types_to_properties(props::DataType...)::Vector{Property}
    check_propertie_types(props...)
    return [PROPERTIES[prop] for prop = props]
end

# register properties macro
macro properties(type::Symbol, ex::Expr)
    return quote
        register_properties($(esc(type)), $ex)
    end
end

# register properties
function register_properties(T::Type, props::Vector{Property})
    # check props
    check_properties_exists(props...)

    # register properties
    @eval properties(::Union{Type{$T},Type{$T{T}}}) where {T} = $props
end

# register properties alternative interfaces
register_properties(T::Type, props::Property...) = register_properties(T, collect(props))
register_properties(T::Type, props::Symbol...) = register_properties(T, collect(props))
register_properties(T::Type, props::Vector{Symbol}) = register_properties(T, [Property(prop) for prop = props])
register_properties(T::Type, props::DataType...) = register_properties(T, collect(props))
register_properties(T::Type, props::Vector{DataType}) = register_properties(T, property_types_to_properties(props...))
register_properties(T::Type, props::PropertyTypes.AbstractProperty...) = register_properties(T, collect(props))
register_properties(T::Type, props::Vector{PropertyTypes.AbstractProperty}) = register_properties(T, [typeof(prop) for prop = props])

# properties function interfaces
properties(::Type{<:AbstractMatrix})::Vector{Property} = []
properties(m::AbstractMatrix) = properties(typeof(m))

# struct
struct TypeA{T<:Number} <: AbstractMatrix{T}
    n::Int

    function TypeA{T}(n::Int) where {T<:Number}
        n > 0 || throw(ArgumentError("$n ≤ 0"))
        return new{T}(n)
    end
end

TypeA(n::Int) = TypeA{Int}(n)

import LinearAlgebra: size, getindex
size(s::TypeA) = (s.n, s.n)
getindex(A::TypeA{T}, i::Int, j::Int) where {T} = 1

@properties TypeA [:symmetric, :inverse, :posdef]
end

module UserModuleB

using ..MyModuleA

export
    TypeC

struct TypeC{T<:Int} <: AbstractMatrix{T}
    n::Int
end

@properties TypeC [:symmetric, :inverse, :graph]
end

# testing codes
using .MyModuleA, .UserModuleB

# reason for Property instead of types
# display(list_properties())
# display(subtypes(PropertyTypes.AbstractProperty))
# display([T() for T = subtypes(PropertyTypes.AbstractProperty)])

"""
use cases:
1. properties of a matrix
2. search matrices by properties
3. define properties for their new matrices
"""

# 1. properties of a matrix
# @show properties(AbstractMatrix)
# @show properties(AbstractMatrix{Int})
# @show properties(Matrix)
# @show properties(Matrix{Int})
# @show properties([1 2; 3 4])
# @show properties(TypeA)
# @show properties(TypeA{Int})
# @show properties(TypeA(5))

# 3. define properties for their new matrices
# display(properties(TypeC))
# @properties TypeC [:symmetric, :inverse, :eigen]
# @properties TypeC [Property(:symmetric), Property(:inverse), Property(:eigen)]
# @properties TypeC [PropertyTypes.Symmetric, PropertyTypes.Inverse, PropertyTypes.Eigen]
# @properties TypeC [PropertyTypes.Symmetric(), PropertyTypes.Inverse(), PropertyTypes.Eigen()]
# display(properties(TypeC))

I would like to highlight a change in register_properties function, which uses Union to accept both type and wrapper types:

@eval properties(::Union{Type{$T},Type{$T{T}}}) where {T} = $props

Also in the macro, type should be escaped:

macro properties(type::Symbol, ex::Expr)
    return quote
        register_properties($(esc(type)), $ex)
    end
end

[abraemer]

@Andy_Zhang

@eval properties(::Union{Type{$T},Type{$T{T}}}) where {T} = $props

Is there a reason why you don’t use the property types here? I would assume that when you use properties in actual code, then it would be best to have it return a tuple of the property types. That way all the information is encoded in the type domain and the compiler can optimize the checks away.

Also I would expect a convenience method for checking whether some object has a certain property. Something like

function has_property(type::Type, property::AbstractProperty)
    return property in properties(typeof(obj))
end
has_property(obj, property) = has_property(typeof(obj), property)

With this check and properties returning a tuple of types, a whole check like has_property(some_obj, Inverse()) would be done at compile time. If you use the symbols, then they still might be constant folded but I think that is a bit more brittle. You’ll need to check this. A clear sign of optimization failure is if you see allocations in the properties related code.

[Andy_Zhang]

@abraemer

Is there a reason why you don’t use the property types here?

See my example code, is just for pretty.

julia> display(list_properties())
9-element Vector{Property}:
 Property(:posdef)
 Property(:inverse)
 Property(:regprob)
 Property(:illcond)
 Property(:graph)
 Property(:symmetric)
 Property(:sparse)
 Property(:random)
 Property(:eigen)

julia> display(subtypes(PropertyTypes.AbstractProperty))
9-element Vector{Any}:
 Main.MyModuleA.PropertyTypes.Eigen
 Main.MyModuleA.PropertyTypes.Graph
 Main.MyModuleA.PropertyTypes.IllCond
 Main.MyModuleA.PropertyTypes.Inverse
 Main.MyModuleA.PropertyTypes.PosDef
 Main.MyModuleA.PropertyTypes.Random
 Main.MyModuleA.PropertyTypes.RegProb
 Main.MyModuleA.PropertyTypes.Sparse
 Main.MyModuleA.PropertyTypes.Symmetric

julia> display([T() for T = subtypes(PropertyTypes.AbstractProperty)])
9-element Vector{Main.MyModuleA.PropertyTypes.AbstractProperty}:
 Main.MyModuleA.PropertyTypes.Eigen()
 Main.MyModuleA.PropertyTypes.Graph()
 Main.MyModuleA.PropertyTypes.IllCond()
 Main.MyModuleA.PropertyTypes.Inverse()
 Main.MyModuleA.PropertyTypes.PosDef()
 Main.MyModuleA.PropertyTypes.Random()
 Main.MyModuleA.PropertyTypes.RegProb()
 Main.MyModuleA.PropertyTypes.Sparse()
 Main.MyModuleA.PropertyTypes.Symmetric()

Use cases do not include checking properties of a type programmatically and dynamically, but will have an interface list_matercies that accepts symbols, Properties, types, or instances of types.

I think it also answers your second concern, we do not really need checks at compile.