The scenario: I am writing a DSLish set of macros that create a few functions that again call code the user has provided verbatim (in the macro call), so something like this:
@mymacro Bla
myfun1()
myfun2()
end
Which will create, say Blafunc1 and Blafunc2 which again call myfunc1 and myfunc2.
Now, all of this works quite nicely. However I find it extremely unsatisfying that my macro clutters the current module’s namespace with new functions with ugly names. My solution would be to let @mymacro create a submodule (Bla in this case) that contains the generated functions which I can then call as e.g. Bla.func1().
All of that works save for one issue which is that I can’t find a general mechanism to make symbols from the surrounding module known within the newly generated module (Bla). I can add a using/import with the parent module name as argument to the generated module, however if the user-supplied functions haven’t been exported they will still not be visible.
So, my question is if there is any way to open the generated submodule’s scope to let it include all symbols in the parent module?
I can think of a few alternative solutions such as parsing the macro arguments for symbols and adding includes for all of them or requiring users to fully qualify all names, but none of them make me happy TBH.
I have a little package called StaticModules.jl that may be appropriate for this, depending on how many things you want to put in your module Bla. Here’s a little example of it in action with your problem:
julia> f(x) = x + 1
f (generic function with 1 method)
julia> g(x) = x - 1
g (generic function with 1 method)
julia> using StaticModules
julia> @staticmodule Bla begin
myfun1() = f(1) - g(1)
myfun2() = f(1) + g(1)
end
StaticModule Bla containing
myfun1 = myfun1
myfun2 = myfun2
julia> Bla.myfun1()
2
julia> @with Bla begin
myfun2() + 1
end
3
What about using gensym for your function names? It means more ugly names but they won’t appear with tab completion for example. Or maybe having the funtions as fields of a named tuple, whose name can be gensymed (that might be similar to the StaticModule solution)
Thanks, that’s a really cool idea.
For my purposes the named tuple solution feels a bit too hacky, though, TBH (not to mention that the @with syntax is a bit inconvenient).
In terms of generating the import statements - I’m not sure I fully understand your code, but I take it get_outers et al parse the AST for non-local symbols, right? I’ve thought about something similar, but this seems to be rather heavy machinery to accomplish something ultimately quite simple.
In any case posting the question and reading the suggestions so far made me realise that maybe I’m on the wrong track anyway.
I have thought about this after reading the suggestions and while I still think it would be nice to have some sort of “open” submodule for things like this, a more idiomatically Julian solution might be to accept the overlap in function names and maybe use traits to tag the difference between different generated versions from repeated macro calls (which might not differ in argument types otherwise).
If you drop the gensym part, I think the suggestion was to make
@mymacro Foo begin
f(1)
g(1)
end
expand to the creation of a NamedTuple like
Foo = (fun1=()->f(1), fun2=()->g(1))
This way, your current namespace is left uncluttered: only Foo is added to it. And users can call Foo.fun1() in the same way as if it had been defined in a submodule.
This is very similar to the StaticModule solution, without any of the complexity associated with finding non-local symbols because it’s not really needed in this case. The following MWE illustrates this:
macro mymacro(name, block)
nt = Expr(:tuple)
counter = 0
for e in block.args
e isa LineNumberNode && continue
counter += 1
funname = Symbol("fun", counter)
push!(nt.args, :($funname = ()->$e))
end
quote
$name = $nt
end |> esc
end
julia> @macroexpand @mymacro Foo begin
f(1)
g(2)
end
quote
Foo = (fun1 = (()->begin
f(1)
end),
fun2 = (()->begin
g(2)
end))
end
Yes, I guess that would work, but I still find it somewhat aesthetically displeasing to have a tuple effectively disguise itself as a module.
One idea I had, though - and this might be what @rfourquet had in mind when talking about gensym’d function names - would be to indeed gensym function names for the generated functions but then assign or call them from functions in a generated module. That way the functions themselves will live in the surrounding namespace without having to import anything while calling the clear-named version is still prefixed with a module name.
I am still debating with myself if I prefer that solution or if my discomfort with naming conflicts is just a remnant of my OOP days and I should just let multiple dispatch (plus possibly traits) sort things out.
It turns out there’s a rather simple solution:
Generate a module containing a noop function for each function you want to have namespaced, then afterwards add methods to these functions (using fully qualified names).
So, as a mockup example:
@genmodule AName
would produce:
module AName
function fun1() end
function fun2() end
end
function AName.fun1(x)
# do something relevant
end
function AName.fun2(y)
# do something else
end
Where fun1(x) and fun2(y) have full access to the outer namespace but can only be called via AName.
That is a nice solution (although I have to admit that I’m a little puzzled why it would be more “aesthetically pleasing” than other solutions here: I would see no reason why modules should be preferred to named tuples in this case… But I guess I’m more of a pragmatist than an aesthete )
In any case, you might want to declare empty generic functions though, in order to avoid creating useless 0-argument methods:
module AName
function fun1 end # Note the absence of parentheses:
function fun2 end # this declares functions with no method
end
AName.fun1(x) = user_supplied_function1(x) # AName.fun1 only has this method
So you want the methods to see variables from the local scope, while the functions they belong to aren’t visible as names in the current scope. What you’ve done seems like a workable and reasonable solution for that.
This is interesting. Can you pinpoint your discomfort with using a named tuple over a module?
Internally a Module is a dictionary-like object with
Name, Parent module
Dynamic variable bindings (affected by eval() and using etc).
Hints for the compiler (optimization level, etc)
But in this case you’re basically not using all these things because you’re using the current module for evaluating the method bodies. From an internals perspective a named tuple seems to give the right thing (effectively a namespace) without providing the unused parts.
Perhaps the discomfort arises because a module is generally used as a container for code (even though code-is-data, and modules can contain non-code bindings). So seeing something of type Module brings certain expectations?
I think you’ve found a compiler bug, could you report it on github?
Aesthetically, I think the main point for me is that the function of a module is specifically to contain code in a separate namespace, which is exactly the functionality I need. Semantically I want a module. Of course I could fake all the functionality I need with a named tuple, but it just doesn’t seem right.
Besides, I don’t know how the compiler and the language are going to develop down the line, so it seems prudent to use a construct that matches the desired functionality as closely as possible.
The other thing is that I might need additional module functionality later on (init function, local variables, selective exports, etc.).
I’ll just chime in and agree with @c42f that there’s not really anything inappropriate about a named tuple here. Holding code is just as appropriate as holding data because code is data.
I also think that you may find that julia’s modules have their own warts and quirks that could make them not so great for your use cases as well. E.g. you can’t create a module in a local scope.
But the funny thing is that you’ve subverted most of the namespacing functionality. To demonstrate:
julia> module Bla
function func1 end
end
Main.Bla
julia> function Bla.func1()
return "hi"
end
julia> parentmodule(Bla.func1)
Main.Bla
julia> methods(Bla.func1)
# 1 method for generic function "func1":
[1] func1() in Main at REPL[25]:1
julia> first(methods(Bla.func1)).module
Main
So while the functionBla.func1 lives in the Bla module, its only implementation method lives in Main. And of course this is correct for your use case, because you want the implementation to see all the names in the scope of @mymacro.
So nearly all of the Module functionalities are being actively subverted here which is why I think this is quite an entertaining use of modules. (Not wrong mind you — I really don’t think it’s that harmful either. Just kind of amusing )
This is an interesting point in its own right. If you return a named tuple, it’d be possible for the whole thing to work fine in local scope. (Where func1 and Blafunc1 may well be closures rather than normal generic functions.) That might not be a use case you care about, however.
This discussion is getting slightly esoteric, but oh well…
I think everybody is making some valid points, but after a bit of thought I think I can say exactly why I don’t like the named tuple solution. There are two reasons:
From a practical point of view a module is preferable as I might need more module functionality (beyond namespacing) down the line. For context: I am writing a set of DSL-like macros for event-based simulations (you can see them in action here). At this point the macros only create the functions that take care of condition testing and rescheduling, but I might want to keep track of some state as well and add a few service functions which again might require imports and exports, etc. At that point having a module would make things easier and tidier.
More fundamentally though, what the macros produce has the syntax of a module. Therefore I expect users will treat it as such and expect module behaviour in all aspects. If I produce a named tuple the difference will leak at some point, confusing users.
The alternative would be to be upfront about it and say “look, this might look and feel like a module, but in fact it’s a named tuple containing functions, so treat it as such”. That would be a valid approach I guess.
However, I think macros increase the mental load on the user anyway as they effectively modify the language itself. So, for each set of macros one has to maintain an internal model of a) how it is used and b) how the code it produces behaves. Many macros are essentially just (very) glorified syntactic sugar, i.e. in principle you could write their results by hand. Part b) becomes much easier if the macro produces code that is similar to what you would end up with if you didn’t have the macros. In my case in absence of my macros I would find it a very reasonable design choice to wrap a simulation in its own module. I would not, however, store the infrastructure functions in a named tuple. This makes modules the lower-friction choice.
All of that said (rather long-windedly), I guess in the end it’s largely a matter of taste which solution you prefer.
)