Create dynamic functions with Macro

General Usage
,
1

Hi,

I’m trying create “dynamic” functions with macro, but it isn’t working. Let-me explain:

I’ve this function:

function f(::Type{T}, message) where T
    println("Type: ", T)
    println("Message: ", message)
end

When I call it f(Int, "Hi"), the output is:

Type: Int64
Message: Hi

Nice! Now I have the dictionary:

map = Dict( Int => "Hello", Float64 => "Hey!", UInt => "Bye" );

And I want dynamic create a function f2(Int), f2(Float64) and f2(UInt) based at map values.

Something like this that works fine, but at this case this f2 will only generate the cases that I manualy added at this code:

macro create_desired(map)
    return esc(Expr(:block, [
                :( f2(::Type{Int}) = f(Int, map[Int]) ),
                :( f2(::Type{Float64}) = f(Float64, map[Float64]) ),
                :( f2(::Type{UInt}) = f(UInt, map[UInt]) ),
                ]...))
end

After @create_desired(map), when I call f2(Int), f2(Float64) and f2(UInt), the output is respectly:

Type: Int64
Message: Hello

Type: Float64
Message: Hey!

Type: UInt64
Message: Bye

If you read at this point, I think do you know what I want. I tried something like this, but it isn’t working:

macro create(map)
    return esc(Expr(:block, [
                :( function f2(::Type{T}) where T
                        return f(T, msg)
                   end
                    for (T, msg) in map )
                ]...))
end

I think it’s a problem with scope at this code. And tried this:

macro create2(map)
    exprs = []
    
    :(
        for (T, msg) in $map
            push!($exprs, function f2(::Type{T}) where T
                                return f(T, msg)
                          end )
        end
    )
                    
    
    return esc(Expr(:block, exprs...))
end

Someone, know how to solve this problem?

2

Here is a simple solution that seems to fit your requirements to me, but it looks pretty different than what you’ve tried so maybe I’m missing something:

mymap = Dict(Int => "Hello", Float64 => "Hey!", UInt => "Bye")

function f2 end

for (t, v) in mymap
  @eval f2(::Type{$t}) = println($v)
end

Is something like this what you are looking for?

1 Like
3

This should work with no eval needed!

function make_f2(mymap)
    # turn it into a NamedTuple so lookup is a constant
    named_tuple_map = (; (Symbol(k) => v for (k,v) in pairs(mymap))...)
    # anon function that looks up the right value
    return function (::Type{T}) where T
        sym_t = Symbol(T)
        msg = named_tuple_map[sym_t]
        return f(T, msg)
    end
end

mymap = Dict(Int => "Hello", Float64 => "Hey!", UInt => "Bye")

const f2 = make_f2(mymap)
4

Hi @cgeoga ,

First, thanks for you answer. I suggested macro because a wanna define how to create the function at one module but create the function at other scope.

Let-me explain this structure:

module B	
	function create_f2(map)
		for (t, v) in map
		  @eval f2(::Type{$t}) = println($v)
		end
	end
	
	export create_f2
end

module A
	using Main.B
	
	map = Dict( Int => "Hello", Float64 => "Hey!", UInt => "Bye" )

	function f2 end
	
	create_f2(map)
	
	f2(Int)
	
	f2(Float64)
	
	export f2
end

I want define how to create f2 at Module B but I want module A own f2 function.

I tried your solution with this struct, but it isn’t working.

5

What are you ultimately trying to do?

This seems like a classic XY problem, where you are reaching for metaprogramming when you should be using higher-order functions or similar.

1 Like
6

Hi @stevengj , I want create functions f2() on a module B, C, D…, but I want define how to create this function at another module A.

Theses modules B, C, D…, uses the same function with the same structure/sequence, but each module have your own implementations of f() function. For example, B can insert an user at DB, C can insert a bank at DB, etc.

@stevengj , the big problem it isn’t generate one function, the real problem is genereate functions for many entities at one call of macro. For example: module B has entities user, sale and payment. I know how to construct these insert functions calling one macro for each entity like:

@create_insert(User, "user")
@create_insert(Sale, "sale")
@create_insert(Payment, "payment")

The problem is:

@create_insert( Dict(User => "user", Sale => "sale", Payment => "payment" ))

I’m trying to do this at webservice REST context to receive requests from the user, but I think is the same problem. At first definition, I simplified the context, because I think the problem happens at multiple contexts and if I coulld define is:

Create a macro that dynamically create k functions by a dictionary/array given at parameter.

7

So write a higher-order function: define a single function f2 in module A, but have it take the function f as a parameter. Then other modules can call A.f2 and pass their own f functions.

1 Like
8

Yeh @stevengj I thought solution, and let me explain why at my case is not a good solution.

At my real context the problem is something like this:

@create_insert_process_request( Dict(User => "user", Sale => "sale", Payment => "payment" ))

And process request is something like:

function insert_process_request(r::Request, ::Type{User})
  check_permission_at_header(r, "user-identifier", "insert-action")
  data = get_post_data(r) #At this point data can be an User
  con = connect_with_db()
  process_request(con, data)
  close_db_conection!(con)
  return Response("Message of success")
end

Supose I’m writing module B with this function insert_process_request(),
note that I have to create a similar function for Sale and Payment. Let me write for Sale

function insert_process_request(r::Request, ::Type{Sale})
  check_permission_at_header(r, "sale-identifier", "insert-action")
  data = get_post_data(r) #At this point data can be an User
  con = connect_with_db()
  process_request(con, data)
  close_db_conection!(con)
  return Response("Message of success")
end

Note this function is almost the same. If these functions happens only at module B, I could write something like this:

function insert_process_request(r::Request, ::Type{T}) where T <: Union{User, Sale, ...}
  check_permission_at_header(r, "PROBLEM-identifier", "insert-action")
  data = get_post_data(r) #At this point data can be an User
  con = connect_with_db()
  process_request(con, data)
  close_db_conection!(con)
  return Response("Message of success")
end

The problem begins at securty identifier, because it’s specific, and can be a very different value. For example the natural is think the entity User has identifier user-identifier, Sale has identifier sale-identifier, etc. But I think that is not health fix this, for example an entitty UserStatus can have a identifier at security system only status-identifier, and I don’t see problem with this differences.

I can create a macro to solve this calling:

@create_insert_process_request(User, "user-identifier")
@create_insert_process_request(Sale, "sale-identifier")
@create_insert_process_request(Payment, "user-identifier")

This code is very ugly, but works. Why I can’t solve this with one macro call?

About a high order function at module A, almost all function at this problem is need at parameter, something like this:

function insert_process_request_highorder(r::Request, identifier::String, action::String, 
get_post_data::Function, connect_with_db::Function, 
process_request::Function, close_db_conection!::Function) 
  check_permission_at_header(r, identifier, action)
  data = get_post_data(r) #At this point data can be an User
  con = connect_with_db()
  process_request(con, data)
  close_db_conection!(con)
  return Response("Message of success")
end

At module B I should define one by one function

insert_process_request(r::Request, ::Type{User}) =
       insert_process_request_highorder(r, "user-identifier", 
         get_post_data_user, connect_with_db_module_B, 
         process_user_request, close_db_connection_module_B!)

insert_process_request(r::Request, ::Type{Sale}) =
       insert_process_request_highorder(r, "sale-identifier", 
         get_post_data_sale, connect_with_db_module_B, 
         process_user_request, close_db_connection_module_B!)

And then I fall into the same problem :smiling_face_with_tear:.

9

Right, so refactor out the common parts. For example:

function insert_process_request(r::Request, request_type::AbstractString)
  check_permission_at_header(r, request_type, "insert-action")
  data = get_post_data(r)
  con = connect_with_db()
  process_request(con, data)
  close_db_conection!(con)
  return Response("Message of success")
end

then call insert_process_request(r, "user-identifier") and insert_process_request(r, "sale-identifier") as needed, or build other abstractions on top of this. No need to repeat the function body.

A key task in software engineering is figuring out what functions and abstractions you need to avoid copy-and-paste code. (Using macros to automate copy-and-paste code — because that’s what they are here — is generally a second-class solution.)

1 Like
10

You could define a single function, e.g.

insert_process_request(r::Request, T::Type) =
       insert_process_request_highorder(r, request_string[T], 
         get_post_data_user, connect_with_db_module_B, 
         process_user_request, close_db_connection_module_B!)

where request_string is a dictionary mapping request types to strings.

(You could also use other abstractions. If you really want to dispatch on the request types, then the request types could be singleton instances of subtypes of an AbstractRequestType, with a method request_string(request_type) that returns the string.)

1 Like
11

@stevengj, the abstract definition of insert_process_request() is at module A, because I need this at module B, C, D, etc.

So I need pass as argument all functions get_post_data::Function, connect_with_db::Function, process_request::Function, close_db_conection!::Function as parameter because they don’t exists at module A, only at other modules.

Let me call A as Microservices, so B can be Ecommerce, C can be CRM, etc.

This is because macro are big deal at this context.

So, that solution I proposed at last answer works but is ugly.

12

See my second post (Create dynamic functions with Macro - #10 by stevengj) — you still don’t need copy-paste code or macros.

13

@stevengj , I saw your answer… looks good, no so beatiful because I fear the “explosion” of functions passed as argument.

I’ll try this at real context and take my observations.

14

If you have an explosion of arguments, it’s probably a sign that you need better abstractions.

Abstraction and software design and refactoring is hard, and I can’t give you a magic formula in a few words that will solve everything — there are whole books written about this. But using macros to automate cut-and-paste code is a last resort, rarely needed for this purpose, and certainly not the first thing you should reach for.

One reason is that macros have to be designed very carefully to avoid subtle fragilities arising from scope. Another is that macros usually have less information to work with than ordinary functions — macros only know how things are “spelled”, whereas functions know types and values. See also the thread: How to warn new users away from metaprogramming

2 Likes
15

Nobody pointed out your conceptual mistake so far:
A macro is not a function that takes some values and return some expression. A macro runs much earlier in the execution timeline and turns code (i.e. raw expressions) directly into other code. As such a macro can never access values of variables or even their types!

Think of a macro as allowing you to define whole new syntax. I don’t think the problem you present here really warrants new syntax. In fact most problems don’t and that’s why writing macros is usually discouraged. For a great usecase for macros you can have a look at e.g. ModelingToolkit.jl or Turing.jl where macros allow an extraordinary efficient way of defining the models/systems.

Of course there are less drastic macros like @test, @info and similar. They are useful because they represent very common patterns in the code and are essentially just templates and fill in the code the user provided. The problem you describe might warrant a small macro to save the user some typing but providing convenience via macros should not be conflated with actually solving the problem via macros. Aim to first solve the problem normally and then consider a macro to provide a nicer interface (this also makes the macro quite simple to write/read/understand/reason about).

2 Likes
16

Hello @abraemer ,

That’s exactly the problem, how to create an easy way to code.

Solving the problem is not my problem. The problem is to make it easier to code the necessary functions.

But I can’t create multiple functions based on the macro parameters.