I have a way to generate a list of strings of lines of code I want to add in a struct. For example, let us say I generated a vector of strings:
FieldDefinitions = ["A :: Float64", "B :: Int64"]
How would I go about adding these lines into a struct definition? Say, making a macro like @insert_lines() which could be used as
struct MyNewStruct
@insert_lines(FieldDefinitions)
end
You probably can write a macro to do that, maybe it need to take the name of the struct and output all the struct in top level. However, I have a very hard time seeing a case this is the most sensible thing to do.
Since macros are evaluated at compile time, it’s a little hard to imagine a place that this would be useful. If you want this so a number of times can all have the same few fields, I would just bundle those fields into a struct and have your typed have one of those. That said, I think this is possible.
Basically, I’m trying to organize a simulation for a bunch of different industrial pieces of equipment. For example, a generic pump will have a number of fields common to all, but some specific types will differ (the system has a piston pump and a centrifugal one). If I’m going to look for a flow rate, I want it to be at the top level and not have to go 3 levels down the struct hierarchy to do it. I could copy/paste all the fields but I don’t want to do that. I just want to “inherit” most of the parent’s fields (but not all, so I can’t do simple inheritance in that case).
I basically made it work. I used a macro generates a list of fields and a list of types (all in String Array format). I then used
CodeLines = join(FieldList .* " :: " .* TypeList, " ; ")
return esc(Meta.parse("begin $CodeLines end"))
I have no idea why I needed the “begin” and “end” for the Meta.parse command. While it’s frustrating to not know why your code dosen’t work, it’s also frustrating to get it working and not know why…
Perhaps this may be helpful?
https://github.com/PetrKryslUCSD/FinEtools.jl/blob/423f8d168b286ebbb16c59196268dd4250f31e95/src/FESetModule.jl#L55
Actually, I ended up seeing something similar to that which helped me. Basically the macro that inserted the lines used the form
macro add_some_fields()
return esc(:(A::Int64; B::Float64))
end
Which worked as expected. The only problem is that my macro produced the fields and the types by looking at a parent object that was an input. Now let us say I wanted to create a pump type CentrifugalPump_Type and inherit all the fields from GenericPump_Type some exceptions (such as specifications). I would have to generate a string representing that code and then parse it. When I used the following command:
InnerExp = :(
A::Int64;
B::Float64; )
I would get
quote
A::Int64
#= none:1 =#
B::Float64
end
I tried using
CodeLines = "A::Int64; B::Float64"
InnerExp = Meta.parse("quote $CodeLines end")
>>
:($(Expr(:quote, quote
#= none:1 =#
A::Int64
#= none:1 =#
B::Float64
end)))
This was not the result I expected, but buried somewhere in the internet, I found someone suggesting a begin … end statement. So I tried this
InnerExp = Meta.parse("begin $CodeLines end")
quote
#= none:1 =#
A::Int64
#= none:1 =#
B::Float64
end
Which yielded the same result as :( A::Int64; B::Float64; ). So I guess putting multiple lines in an expression is implicitly assuming a “begin…end” statement. This is what ended up working in the simple macro.
macro add_some_fields()
CodeLines = "A::Int64; B::Float64"
return esc(Meta.parse("begin $CodeLines end"))
end
Now that CodeLines are strings, I can very easily generate any kind of code that I want.
It should never be necessary to manipulate Julia code as strings–doing so introduces lots of annoying edge cases and parsing issues, as you’ve just discovered. In your case, you can skip all of the parsing and the manual begin and end splicing by just writing an expression directly:
julia> CodeLines = :(A::Int64; B::Float64)
quote
A::Int64
#= REPL[35]:1 =#
B::Float64
end
julia> @eval struct Foo
$CodeLines
end
julia> f = Foo(1, 2)
Foo(1, 2.0)
julia> f.A
1
julia> f.B
2.0
No parse, no begin, no string interpolation, just working code with less typing!
This is a really important rule of thumb in Julia metaprogramming: Use expressions; don’t use strings.
So let us say I have a type whose fields I want to duplicate in another type. I came up with this:
struct P
a::Int64
b::Float64
end
dump(:(struct P
a::Int64
b::Float64
end))
macro duplf(P)
ex = Expr(:block)
return quote
global ex = Expr(:block)
for (n,t) in zip(fieldnames($P), fieldtypes($P))
global n, t
global ex
push!(ex.args, LineNumberNode(0, ""))
push!(ex.args, Expr(:(::), Symbol(n), Symbol(t)))
end
ex
end
end
dump(@duplf(P))
struct OP
@duplf(P)
end
Unfortunately, even though the macro duplf returns an expression which matches the definition of the fields of the type P, the definition of the type OP does not work and there are no fields in that structure. I must be missing something.
It gets really hard to use expressions for what I’m doing, especially once the types have a parameter in them (my types are basically like arrays, that can have strings, floats, or even complex and dual numbers). I basically got something like this working
macro inherit_asset_fields(ParType_S::Symbol, T::Symbol)
ParType = eval(ParType_S)
#Exclude "Specs" from the duplication, specific to "Asset" inheritance
IndKeep = findall(x-> x != :Specs, fieldnames(ParType))
#Creates an arbitrary sentinel type to recover parameterization
SentinelType = Some{T}
#Get the fields lists and types (with sentinel types)
FieldList = string.( fieldnames(ParType)[IndKeep] )
TypeList = string.( fieldtypes(ParType{SentinelType})[IndKeep] )
#Recover the parameterization through the sentinel type
TypeList = replace.(TypeList, string(SentinelType)=>string(T) ) #
CodeLines = join(FieldList .* " :: " .* TypeList, " ; ")
return esc(Meta.parse("begin $CodeLines end"))
end
mutable struct MyOldType{T}
Name :: Symbol
A :: T
B :: T
V :: Vector{T}
Specs :: String
end
mutable struct MyNewType{T}
@inherit_asset_fields(MyOldType, T)
C :: T
NewSpecs::String
end
I know this is inelegant, but I’m not sure where I start in terms of learning how to use expressions effectively, or how I go about recovering parameters from fieldtypes(MyOldType) without some dumb hack like creating Some{:T} and doing a string replace on it. If I try using fieldtypes(MyOldType), I just get “Any” where T could be. The problem is, the actual struct might actually include a field of type “Any” (I’ll try to avoid it, but it might end up happening somehow, even by accident).
This seems to work:
macro dup_fields(T)
structType = Core.eval(__module__, T)
ex = Expr(:block)
for fieldName in fieldnames(structType)
fieldType = fieldtype(structType, fieldName)
push!(ex.args, Expr(:(::), fieldName, Symbol(fieldType)))
end
return esc(ex)
end
I’m not really happy about the eval, but not sure how to get around this?
struct P
a::Int64
b::Float64
end
struct Q
@dup_fields(P)
end
dump(P)
dump(Q)
I don’t think there is a way to get around the eval. Without the “eval”, the macro doesn’t even know if the type exists yet so it can’t execute “fieldnames” if it can’t be evaluated at the module level; it needs to already exist in the module. Anyway, your code nearly worked in my case. I modified it to suit mine
macro inherit_asset_fields(AssetType_S, T)
BaseAssetType = Core.eval(__module__, AssetType_S)
AssetType = BaseAssetType{T}
ex = Expr(:block)
for fName in filter( x-> x!=:Specs, fieldnames(AssetType) )
fType = fieldtype(AssetType, fName)
push!(ex.args, Expr(:(::), fName, fType))
end
return esc(ex)
end
mutable struct MyOldType{T}
Name :: Symbol
A :: T
B :: T
V :: Vector{T}
Specs :: String
end
mutable struct MyNewType{T}
@inherit_asset_fields(MyOldType, T)
C :: T
NewSpecs::String
end
It ALMOST works, however, it gets stuck on Vector{T}. It actually tries to put in Vector{:T} when it should actually put in Vector{T}. I’m not sure how to get the symbol :T to be converted to just T in the push! Expr statement without converting fType into a string and doing a search/replace on a sentinel type. In every other instance it gets handled automatically and gracefully without string conversion.
Right, the quote needed for the interpolation is the problem.
Use composition (search for “composition over inheritance”).
Trying to replicate what inheritance does is fighting the language.
Okay, I think I found the root of this problem. If I try to get an instance of a type by using
MyOldType{:T}
I will get for field :V
Vector{:T}
Which is actually a Vector with the type parameter :T. However, what I actually want to do is convert this “Type” into an “Expression” of Vector with an argument T. Converting this type into a string and re-parsing it will correctly reinterpret this as an expression. Is there any way to do this without such a dumb hack?
Okay! I managed to find a remarkably clean way to do this!! I’ve also seen other threads of people who want to do something similar (and it turns out there is a Mixers.jl package that can be used to do what it is that I’m doing, but in my opinion, this method looks more intuitive). Anyway, the key step was to infer whether the FieldType was a Symbol, a non-parametric type or a parametric type. I also learned I could get the actual constructer of a type from the “.name.wrapper” field and get the parameters from the “.parameter” field too. Once this “get_type_expression” function worked, everything else fell into place.
function get_type_expression(X::Union{Symbol,DataType})
if X isa Symbol
return X
elseif X isa DataType
Params = X.parameters
if isempty(Params)
return Symbol(X.name.wrapper)
else
return Expr(:curly, Symbol(X.name.wrapper), Params...)
end
end
end
macro inherit_asset_fields(AssetTypeExp::Expr)
BaseAssetType = Core.eval(@__MODULE__, AssetTypeExp.args[1])
TypeSymbols = AssetTypeExp.args[2:end]
AssetType = BaseAssetType{TypeSymbols...}
ex = Expr(:block)
for fName in filter( x-> x!=:Specs, fieldnames(AssetType) )
fType = fieldtype(AssetType, fName)
fTypeParsed = get_type_expression(fType)
push!(ex.args, :( $fName :: $fTypeParsed ) )
end
return esc(ex)
end
mutable struct MyOldType{T}
Name :: Symbol
A :: T
B :: T
V :: Vector{T}
Specs :: String
end
mutable struct MyNewType{T}
@inherit_asset_fields(MyOldType{T})
C :: T
NewSpecs::String
end
OP I think you may be missing the forest for the trees a bit here. I think the solution to your problem is to not use getproperty to access fields but rather to define functions that retrieve nested parameters. Something like the following
julia> struct IntersectType{T}
x1::T
x2::T
end
julia> abstract type SpecificType end
julia> struct SpecificType1 <: SpecificType
i::IntersectType
y1
end
julia> struct SpecificType2 <: SpecificType
i::IntersectType
y2
end
julia> function x1(s::SpecificType)
s.i.x1
end
x1 (generic function with 1 method)
julia> function x2(s::SpecificType)
s.i.x2
end
x2 (generic function with 1 method)
So you have an IntersectType that stores the intersection of all parameters. Then you have a variety of structs part of the same abstract type.
Then for each shared parameters, you create a function that retrieves the nested parameters.
It should be feasible to do this programmatically using a strategy similar to what we use for Missing here.
EDIT: I should add that in general, replacing getproperty calls with functions is generally considered good programming style, as it means you don’t have to expose the internals of your structs to the user.
I actually thought about doing it this way, but in the end, I realized that the actual structure is informative of the object. There are two hierarchical relationships that are going on, one is taxonomic and the other is componential, so using composition becomes confusing as it convolutes the intent behind the object’s structure. Also, since this is a simulation with all the internals having physical significance, I actually WANT to expose all the internals to the user.
It turns out that multiple dispatch really helps here when I want to predict the behaviors of two different kinds of pumps. I can mix and match any analysis tools I want in a larger function as long as I know it has all the appropriate fields. Trying to call one analysis package on a “parent” type would work in almost all cases, but I found a couple where such a paradigm would end up being too restrictive. Julia really saved my figurative butt here. I’d be very hard pressed to implement this in a pure OOP paradigm.
You can have nested abstract types, if a few cases are too restrictive you can always define more abstract or concrete types to allow for special behavior.