I am trying to wrap my head around metaprogramming and macros and I struggle to figure this out and I think I am on the wrong path. I have the feeling that I need to build up an expression tree programmatically instead of the current approach, but let’s see.
To avoid an XY problem I’d like to provide a not so MWE (see the end) since there might be a better way to do what I am up to.
My specific problem is that I want to generate a function call with an arbitrary number of arguments. In my macro I have access to a list of types in form of a Vector{Symbol}
like this:
types = Symbol[:Int32, :Float32] # the length of this varies
which I’d like to turn into (dummy example):
foo() = Foo(Int32, Float32)
I managed to do it but the problem is that I am calling eval()
which is then part of the function definition and I lose a lot of performance. It’s looking something like this inside my macro definition:
types = Symbol[:Int32, :Float32]
@eval foo() = Foo([eval(t) for t in $(types)]...)
This is of course less performant then writing
@eval foo() = Foo($(types[1]), $(types[2]), etc.)
Long story short, here is my code:
using BenchmarkTools
macro io(data, parameters...)
eval(data)
struct_name = data.args[2]
fields = filter(f->isa(f, Expr), data.args[3].args)
types = [f.args[2] for f in fields]
function_args = [:(:call)]
@eval unpack(io, ::Type{$(struct_name)}) = begin
# hardcoded version as proof of concept, this works and is super fast
$(struct_name)(ntoh(read(io, $(types[1]))), ntoh(read(io, $(types[2]))))
# this works too but "bakes in" the `eval()` into the function definition -> super slow
#$(struct_name)([ntoh(read(io, eval(t))) for t in $(types)]...)
end
end
@io struct Foo # this struct could have any number of fields
a::Int32
b::Float32
end
# a huge buffer as random data pool for testing
buf = IOBuffer(rand(UInt8, sizeof(Foo)*100000000))
@btime unpack($buf, Foo) # gives ~3ns for the hardcoded and ~800ns for the dynamic `eval`
The @io
macro essentially takes a struct, evaluates it and additionally it creates a function
unpack(io, ::Type{StructName}) = StructName(ntoh(read(io, TypeOfField1)), ntoh(read(io, TypeOfField2)), etc.)
I hope it’s clear.
I also tried to build up the entire call tree using nested Expr
but it seemed too complicated to be true
A little bit of background: I am doing this function generation since according to my performance studies, this is by far the fastest way of parsing big endian data. I thought calling ntoh()
and read()
multiple times would have a huge impact on the performance but it seems that Julia and LLVM do some magic there. I also tried StructIO
from Keno but it’s way slower (around 1us).
Thanks in advance for your time!