Lexically Scoped Macros?

Internals & Design
,
1

Hi, I recently had a conversation on the lisp subreddit where I was convinced that allowing for lexically scoped macros would be a potentially interesting and useful addition to Julia.

My questions are

  • Why aren’t lexically scoped macros allowed?
  • Is there a possibility of implementing them in the future along with an option to pass an environment parameter to the macro expander?
2 Likes
2

I heard from someone here that Lexical macros were excluded for performance reasons as they make it “impossible to statically analyse the code and generate fast code”. This makes no sense to me as lexically scoping the macro should only serve to further localize its effects and should be converted to proper code at before compilation so I fail to see any problem.

3

I haven’t done any work in Lisp, but I tried reading the docs on lexically scoped macros. I’m having trouble understanding how they’re different from the macro hygiene that Julia already does by default: https://docs.julialang.org/en/stable/manual/metaprogramming/#Hygiene-1

Do you have a sense of what a lexically scoped macro could do that Julia macros can’t?

4

I’m having trouble understanding how they’re different from the macro hygiene that Julia already does by default

Macros in Julia are only allowed to be defined in the global scope. What this means is that I am not allowed to build a function or macro which defines a macro:

julia> function make_greeting_macro(greeting)
           macro geet(name)
                  return :( println(greeting, $name) )
           end
       end
ERROR: syntax: macro definition not allowed inside a local scope

Wanting to make macros or functions that make macros is a pretty rare use case but its not unimaginable.

Do you have a sense of what a lexically scoped macro could do that Julia macros can’t?

In the thread I linked to in my original post, the lisper who replied to me showed a neat macro he made which is passed a function and a list of symbols. The macro then returns the original list with the specified function applied only to the variables which were unbound.

I doubt such functionality is impossible to replicate in Julia but it certainly is impossible the way he did it. His macro works by defining a new macro in that local context which is passed an environment variable which contains information about the local scope and the variables which are defined there.

To be clear, I don’t really know how useful these lexical macros actually are but I’d like to understand why we don’t allow them and if there isn’t a good reason to allow them I think they should be allowed even if we can’t currently divine a super critical use for them right now.

5

My guess would be that local macros are not a priori incompatible with Julia, but adding them is not a high enough priority at the moment to justify the work.

I end up writing macros much less than I did in Common Lisp. In CL, the argument is that you can do anything with macros, but from another perspective, sometimes you have to use macros to paper over cracks of the language (with a standard that was frozen decades ago, etc). I use great macro libraries written by others (eg Parameters.jl, MacroTools.jl), but my answer to the “do I really need a macro” question is frequently no, much more than in CL.

I find that generated functions are extremely powerful and work fine for many of the use cases I would have used lexical macros before.

1 Like
6

Macros can make macros (albeit still in global scope).

7

Got it, thanks. I suppose I can see how that might be useful, but, as @Tamas_Papp says, I’ve generally found that passing and returning functions is a better solution for my use cases. So I would do:

function make_greeting(greeting)
  function greet(name)
    println(greeting, name)
  end
end
8

yes, I suspect you’re probably right and that this feature isn’t badly needed and so isn’t really worth the development time. I was mostly curious about if there was any performance reason or something why it’s not reasonable to define macros outside the global scope.

Out of curiosity, do you think that it’s possible to make a macro which lets one make a lexical macro? Like some macro @lexical_macro such thatcher following:

Function foo()
    @lexical_macro bar()
        ...
    End
End

Is allowed and @bar only exists inside the function foo?

I was thinking this might be a fun project for me to try and puzzle out but I’m not sure how to start.

9

Obviously for the silly example I used there is no reason to use a macro anywhere. That does not mean in principal that a function will always do the job you want a macro to do though. I get that this is probably an extremely niche thing though.