# Metaprogramming - defining a function

I’m trying to build a function using metaprogramming. I followed this approach, which is good.

I am defining a function similar to the following:

``````function myfunc(x::Int64,....)
# Now I call another function using x as a parameter
anotherfunction(x)
end
``````

In anotherfunction, I have something like

``````function anotherfunction(x)
if typeof(x) <: Int
#whatever
...
``````

This fails because anotherfunction receives x as a Symbol, rather than the value.

The actual functions are (sorry about my messy code):

``````function create_function( ptr::Ptr{VSPlugin}, funcname::String, params)
#https://stackoverflow.com/questions/37230071/metaprogramming-julia-functions-with-args-and-keyword-args
#https://github.com/JuliaMath/DecFP.jl/blob/master/src/DecFP.jl

# Creamos la lista de parámetros: los obligatorios + los opcionales (les asigna nothing)
# Lo siguiente crea la interfaz que queremos que tenga nuestra función.
f_arguments = []
f_argumentsopt = Expr(:parameters)
for i in 1:length(params)
param = params[i]
tipo = get_new_type( param[2] )
if length(param) == 2 #Expr(:kw,:y,Float64(2))
ex = Expr(:(::),Symbol(param[1]),tipo)
f_arguments = [f_arguments ; ex ]
elseif length(param) > 2 && param[3] == "opt"
# param=Union{Int64,Nothin}=nothin
ex = Expr(:(::),Symbol(param[1]),Union{tipo, Nothing}) # Asignamos el tipo
ex = Expr(:kw,ex, :nothing)
#ex = Expr(:kw,Symbol("\$(param[1])"),Union{tipo, Nothing}=nothing)
f_argumentsopt.args = [f_argumentsopt.args; ex ]
end
end
f_call = Expr( :call, Symbol(funcname), f_argumentsopt )

# Añadimos sólo los parámetros obligatorios
for ex in f_arguments
f_call.args = [f_call.args ; ex]
end

# Cuerpo de la función
f_body = quote
# Creamos un vsmap a partir de la llamada.
lista = []
#mymap = createMap()
for param in \$params
if length(param) == 2  # Los obligatorios
tmp = (param[1], Symbol(param[1]) )
#if param[2] == "int"
#    propSetInt( mymap, param[1], eval(Symbol(param[1])), paAppend )
#end
#print(param[2])    # Una opción construir el vsmap a partir del tipo definido: int llama a propSetInt
lista = [lista ; tmp]
elseif length(param) > 2 && param[3] == "opt"
for (k,v) in kwargs
if k == Symbol(param[1]) && !(v == nothing)
tmp = (param[1], v)   # Añadimos el parámetro con su valor
lista = [lista ; tmp]
end
end
end
end
println("LISTA2VSMAP: \$(lista) ") # Toma el tipo del valor, que es desconocido.

vsmap = list2vsmap( lista )

invoke( \$ptr, \$funcname, vsmap )
end

f_declare = Expr( :function, f_call, f_body )

eval( f_declare )
#f_declare
end
``````

and the call to lista2vsmap is:

``````function list2vsmap( items ) #::Array{Any,1}
vsmap = createMap()
for item in items
key = item[1]
value = item[2]
#if typeof( value ) == Symbol
#    value = esc(eval(value))
#    println(value)
#end
if typeof( value ) <: AbstractFloat
propSetFloat( vsmap, key, value, paAppend )
elseif typeof( value ) <: Int
propSetInt( vsmap, key, value, paAppend )
elseif typeof( value ) <: Array{Int64,1}
propSetIntArray( vsmap, key, value )
elseif typeof( value ) <: Array{Float64,1}
propSetFloatArray( vsmap, key, value )
elseif typeof( value ) <: AbstractString
propSetData( vsmap, key, value, paAppend )
elseif typeof( value ) == VSNodeRef
propSetNode( vsmap, key, value, paAppend )
elseif typeof( value ) == VSFrameRef
propSetFrame( vsmap, key, value, paAppend )
elseif typeof( value ) == VSFuncRef
propSetFunc( vsmap, key, value, paAppend )
else
println("[ERROR] vsmap - list2vsmap: tipo no soportado: \$(typeof(value))")
end
end
vsmap
end
``````

Thanks

Posting a minimal working example may help lower the barrier for people willing to help but won’t go through so much code.

4 Likes

I am a beginner also, so my little bit of knowledge could be a dangerous thing

Is there a reason to do all of the type tests instead of converting this to a multiple dispatch function, writing just the method for each type separately? (And let the compiler do the testing).

If not, then it still leaves the “how”

``````fakeshow(x) = show(x)
fakeshow(x::String) = "arg is a string '\$x'"
fakeshow(x::Number) = "arg is 1 less than: '\$(x + 1)'"``````
3 Likes

There are two important reasons: I come from python and I’m also a newbie!! ;oP So, yes you are right I have changed the code to Julia’s ways.

I still have the problem nonetheless. I’ll try to think on a minimal example.

This is a working example. I was trying to reproduce the problem that I have and I am not managing:

``````function create_function( funcname::String)
f_call = Expr( :call, Symbol(funcname),  Expr(:(::),Symbol(:x),Int64) )

lista = :x

f_body = quote
newvalue = list2vsmap( \$lista )
return newvalue
end

f_declare = Expr( :function, f_call, f_body )

eval( f_declare )
end

function list2vsmap( x::Int64)
x * 2
end

f = create_function( "myfunc" )
println("Testing my created function: \$(f(2))")
``````

In this short example, I was expecting list2vsmap to complain and say:

``````ERROR: LoadError: MethodError: no methoc matching list2vsmap(::Symbol)
``````

but this short example works. While my other code doesn’t.

To give further insight, the key seems to be in the following bit of code that I have commented:

``````    f_body = quote
# Creation of a VSMAP (kind of a dict)
lista = []   # lista stands for "list" in Spanish
for param in \$params
if length(param) == 2  # This is mandatory arguments in f_call
tmp = (param[1], Symbol(param[1]) ) # Creates pairs such as: ("level", :level)
lista = [lista ; tmp]
elseif length(param) > 2 && param[3] == "opt"  # This is for optional arguments in f_call
for (k,v) in kwargs  # We check among the keywords arguments
if k == Symbol(param[1]) && !(v == nothing)  # Only keywords not set to nothing are considered
tmp = (param[1], v)   # Añadimos el parámetro con su valor
lista = [lista ; tmp]
end
end
end
end

if \$funcname == "SetLogLevel"
eval(println(lista))      # This displays: Any[("level", :level)]
end

vsmap = list2vsmap( lista )
invoke( \$ptr, \$funcname, vsmap )

end
``````

When I execute `eval(println(lista))` I am getting what I want: a list of pairs where I have the name of the param and a Symbol whose value should come from the function call.

Later, list2vsmap looks as follows:

``````function list2vsmap( items ) #::Array{Any,1}
vsmap = createMap()            # Creates a VSMAP (it is like a dictionary in "C")
for item in items              # Iterates on the provided list of pairs; in the example is Any[("level", :level)]
key = item[1]              # This is the "key"
value = item[2]            # This is the value (a Symbol representing the function argument such as :level)
setvalue(vsmap, key, value)  # Depending on the type of _value_ should call one method or another
end
vsmap
end

setvalue(vsmap::VSMap, key::String, value::Int) = propSetInt( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value::AbstractFloat) = propSetFloat( vsmap, key, value, paAppend )
....
``````

So list2vsmap converts the list into an VSMAP (see the comments).

What I don’t understand is why in this case setvalue is finding that value is a ::Symbol while the short example above, finds that x is an Int64.

I have checked that the signature of the function created here looks as expected:

``````SetLogLevel(level::Int64;)
``````

I have found where is the issue, but I don’t know how to solve it. The problem is in the definition of the function’s body (f_body).

I am creating a function with the signature:

``````SetLogLevel(level::Int64)
``````

and I am calling it using `level=1` as follows:

``````SetLogLevel(1)
``````

When I modify `f_body` as follows:

``````    f_body = quote
if \$funcname == "SetLogLevel"
eval(println("TEST1: \$(level)"))                   # OK: TEST1: 1
tmp = Symbol("level")
eval(println("TEST2: \$(tmp)   \$(typeof(tmp))" ))   # OK: TEST2: level   Symbol
eval(println("TEST3: ", eval(tmp) ))               # NOK: ERROR: LoadError: UndefVarError: level not defined
end
``````

it can be seen that:

• TEST1: the variable `level` is seen within `f_body` and it values `1`.
• TEST2: it creates `Symbol("level")`; we can see it represents `level` and the type is `Symbol`.
• TEST3: when I evaluate `Symbol("level")` it is not finding it in this context.

So what am I doing wrong? What do I have to do to get the value of the argument in `f_call`? Why `level` works while `eval(Symbol("level"))` doesn’t?

Note: I have just check that:

``````tmp = \$(Symbol("level"))   #  In this case, tmp gets the value that I want: 1
``````

but how can I get it from:

``````tmp = Symbol("level")
``````

because `\$(tmp)` doesn’t work.

`eval` works in global scope, so any variables used in the expression evaluated must be globally defined. This makes it a bit confusing to reason about plus it is usually not the best way to do what you want to do. The following example is probably what you are seeing:

``````julia> function f(a)
eval(:a)
end
f (generic function with 1 method)

julia> f(1)
ERROR: UndefVarError: a not defined
Stacktrace:
[1] top-level scope
[2] eval at .\boot.jl:319 [inlined]
[3] eval(::Symbol) at .\client.jl:389
[4] f(::Int64) at .\REPL[1]:2
[5] top-level scope at none:0

julia> function f(a)
eval(:(a = \$a))
end
f (generic function with 1 method)

julia> f(1)
1
``````

In the first case, we are just calling `a` in global scope which is not defined. In the second case, we are calling `a = 1` in global scope so we are defining `a`. If you type `a` in the REPL then you will get 1. I hope this helps.

1 Like

I’m still not getting a solution.

I am looking for the following:

``````julia> function f(a)
local m = :(\$a)
m
end
f (generic function with 1 method)

julia> f(1)
1
``````

but when I replace `a` with `Symbol("a")`:

``````julia> function f(a)
local m = :(\$(Symbol("a")))
m
end
f (generic function with 1 method)

julia> f(1)
:a
``````

What do I have to do with `Symbol("a")` in order to get `1` instead of `:a`?

You can’t.

2 Likes

The only way you could get to `a` from its symbol is evaling, but `eval` works in the global scope, so that would look for an `a` in the global scope and not in your function and not do what you want.

Maybe you should explain with a bit more details what you are trying to do (wrapping a library?), because it looks like there’s maybe something not quite right with the way you are approaching your problem (this metaprogramming business can be tricky).

Now we are getting somewhere. Now, I know I cannot follow this path.

I am wrapping a C library (Vapoursynth). I am creating a number of functions trying to keep a similar signature to the C ones.

I am invoking the C functions with a VSMap which is kind of an array of key/values pairs in C. I get those arguments through the function signature. But I am not managing to get the functions arguments values when I am referencing them as symbols.

Maybe push and pop the values to a global vector, to keep the scope global? I don’t know how it translates to your function generator exactly, but consider:

``````A = Vector{Any}()
function f(a)
push!(A, a)
eval(pop!(A))
end

f(1)
``````

Added: of course, you cannot push a Symbol of a local reference and still have the pop!ed Symbol have that local reference in global context. You have to pass the local value with push! and pop!.

Just out of curiosity, what’s the point of being able to define a function call programmatically if then you cannot refer to its arguments?

I don’t know what “refer to its arguments” mean.

Here is an example of some functions generated dynamically for reference

Dynamically generating functions just saves you from writing down a bunch of code that looks very similar (in the link above for 4 different number types), but in the end there is nothing special about a function that was generated from metaprogramming or by writing the body by hand.

So if you could just write down a few of the functions you want to generate by hand and then it should be easier to see how it can be done programatically. Preferably a bit simpler than in the first post of this thread.

1 Like

Going back to your original question, if you are trying to create a function based on the argument types that are only known at run time, consider using generated functions.

I’ll try to explain better.

In the MKLSparse.jl example, the function signature is known. So you can use its arguments without problem. For example, line 35 displays argument “transa” which is later used in lines 38 and 39.

In my example, I am creating the function signature programatically (see the creation of f_call in my first post). Later I try to use those arguments creating symbols. Something like:

``````Symbol(param[1])
``````

The documentation says:

In the context of an expression, symbols are used to indicate access to variables; when an expression is evaluated, a symbol is replaced with the value bound to that symbol in the appropriate scope.

I think this post is clear regarding what I am trying to do. The MKLSparse.jl example fits on the first case of the post.

So I would reword my question as:

What is the point of being able to define a custom function signature if later I won’t be able to use the values of its arguments in the function’s body?

In fact, I would say that documentation (or the documentation) might be wrong:

``````julia> a = "hello"
julia> function m(a)
Symbol("a")
end
julia> m(1)
:a
julia> eval(m(1))
"hello"
``````

I repeat what the documentation says:

[…] a symbol is replaced with the value bound to that symbol in the appropriate scope.

I would say that the scope of `Symbol("a")` would be the function. And if not, how could I refer to the argument named “a” (bearing in mind that I am creating a bunch of functions and that the argument name is different in each case).

I am not sure if this is more clear now.

I changed this:

``````function list2vsmap( items ) #::Array{Any,1}
vsmap = createMap()
for item in items
key = item[1]
value = item[2]
#if typeof( value ) == Symbol
#    value = esc(eval(value))
#    println(value)
#end
if typeof( value ) <: AbstractFloat
propSetFloat( vsmap, key, value, paAppend )
elseif typeof( value ) <: Int
propSetInt( vsmap, key, value, paAppend )
elseif typeof( value ) <: Array{Int64,1}
propSetIntArray( vsmap, key, value )
elseif typeof( value ) <: Array{Float64,1}
propSetFloatArray( vsmap, key, value )
elseif typeof( value ) <: AbstractString
propSetData( vsmap, key, value, paAppend )
elseif typeof( value ) == VSNodeRef
propSetNode( vsmap, key, value, paAppend )
elseif typeof( value ) == VSFrameRef
propSetFrame( vsmap, key, value, paAppend )
elseif typeof( value ) == VSFuncRef
propSetFunc( vsmap, key, value, paAppend )
else
println("[ERROR] vsmap - list2vsmap: tipo no soportado: \$(typeof(value))")
end
end
vsmap
end
``````

into this:

``````function list2vsmap( items ) #::Array{Any,1}
vsmap = createMap()
for item in items
key = item[1]
value = item[2]
setvalue(vsmap, key, value)
end
vsmap
end

setvalue(vsmap::VSMap, key::String, value::Int) = propSetInt( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value:: Array{Int64,1}) = propSetIntArray( vsmap, key, value )
setvalue(vsmap::VSMap, key::String, value::AbstractFloat) = propSetFloat( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value:: Array{Float64,1}) = propSetFloatArray( vsmap, key, value )
setvalue(vsmap::VSMap, key::String, value::AbstractString) = propSetData( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value::VSNodeRef) = propSetNode( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value::VSFrameRef) = propSetFrame( vsmap, key, value, paAppend )
setvalue(vsmap::VSMap, key::String, value::VSFuncRef) = propSetFunc( vsmap, key, value, paAppend )
``````

I am still learning!

What do I have to do with `Symbol("a")` in order to get `1` instead of `:a` ?

Perhaps?

``````f(a) = a
``````

or less trivially and maybe more useful:

``````x = :a
@eval begin
f(\$x) = \$x
end
``````

or with a macro:

``````macro f(x)
return esc(x)
end
``````

called using `@f(x)` or `@f x`.

Also for sake of completeness, where you want to generate the body of `f` for different types of `a`, you can do:

``````@generated function f(a::T) where T
if T <: Integer
return :(a)
else
return :(error("Sorry, it is not clear what you want to do with the function's argument."))
end
end
``````

This is pretty much every way to define the behavior of `f` in Julia, see what suits you best.

Edit: the generated function was an overkill for the example above and should not be used in these cases. This is nothing more than a toy example.

There is still no need for any inner evals.

For example

``````function_name = :foobar
sig = (Int, Float64, Int32)
variables = [:a, :q, :d]
body = :(a + q * d)

function create_function_expr(function_name, sig, variables, body)
Expr(:function,
Expr(:call,
function_name,
[Expr(:(::), s, t) for (s, t) in zip(variables, sig)]...),
body
)
end
``````

and using it:

``````julia> create_function_expr(function_name, sig, variables, body)
:(function foobar(a::Int64, q::Float64, d::Int32)
a + q * d
end)

julia> eval(create_function_expr(function_name, sig, variables, body))
foobar (generic function with 1 method)

julia> foobar(1, 2.0, Int32(3))
7.0
``````
1 Like