It seems to me that macros are defined much too often in the Julia ecosystem and the (error-prone) definitions of such macros are often very complex. FWIW, the Julia Manual page on metaprogramming already heavily discourages defining macros: eval and defining new macros should be typically used as a last resort. It seems to me that (almost?) the only good way to design a package that includes macros is like this: the macro simply provides some syntax sugar over an actual Julia function, with only minimal logic used in the definition of the macro itself if the macro is part of the public API, the function that the macro wraps should also be public (with public or export, in future Julia versions) and documented Even if these rules were followed, each macro introduces its own syntax on top of Julia, causing extra cognitive load for the programmer and trouble for tooling, meaning that both macro usage and macro definition should perhaps be discouraged to a degree within public packages, even if a macro/DSL is sometimes really necessary. One way of discouraging unnecessary definition of macros would be to disable automatic merging (automerge) for General registry packages when the package includes a macro definition (perhaps with the restriction that the macro name is public instead of just internal). Another potentially good restriction would be to whitelist macros that define new string literals. @ShalokShalom suggests that it may be better to just give some kind of warning/tag. A more conservative approach would perhaps be to only do this when the definition of the macro crosses some complexity threshold. Are these ideas good?

Me and nsajko launched our posts simultaneously, can somebody please merge them? [image] General Registry: Create tag for macro usage New to Julia In a recent discussion about newbie friendliness, the topic about macros came up. Someone suggested, that it may be beneficial to prevent auto merges to the general registry, for code that uses them. I think that’s a bit too harsh. But what about creating a tag, so that we can review the PR, and possibly comment on the overuse of it? Macros are overused in nearly every language that uses them, and I think this could be a way to deal with this issue. Eventually might even the compiler be able…

[image] nsajko: Are these ideas good? This seems very arbitrary. Almost any feature can be misused and punishing someone in this kind of way for using a language feature feels off.

We don’t even block automerge for packages that have no tests. We don’t enforce standards at the package registry level. I can imagine we might start to do some basic ones, like the aforementioned tests being required, but I can’t imagine we will ever have the something as subjective as this in the General Registry CI tests.

I mostly push back on excessive macro use by using this quote from @stevengj 's 2019 JuliaCon keynote. Functions are mostly good enough for Jeff Bezanson. Don’t try to outsmart Jeff Bezanson. [slide conveying above information] Video: https://youtu.be/mSgXWpvQEHE?t=578

[image] nsajko: One way of discouraging unnecessary definition of macros would be to disable automatic merging (automerge) for General registry packages when the package includes a macro definition (perhaps with the restriction that the macro name is public instead of just internal). Another potentially good restriction would be to whitelist macros that define new string literals. @ShalokShalom suggests that it may be better to just give some kind of warning/tag. A more conservative approach would perhaps be to only do this when the definition of the macro crosses some complexity threshold. :-1:

blocking automerge is a very heavy handed approach to influencing the ecosystem and creates a lot of manual work for registry maintainers. I think it should be used very sparingly, and something like defining a macro is a totally legit thing for a package to do. IMO if you want to influence folks you need to convince them, e.g. write blog posts showing how some particular code that uses macros would be better without it (or PR such code). But your case will need to be solid to actually change someone’s mind.

To me this seems plausible, at least to mark the registration PR with a macro label and/or extend the the auto-merging period. [I’ve never second-guess anyone on adding a macro, also since I might not know, I have blocked some package registrations for other reasons, and occasionally look at code before registered.] I’m not sure I agree with the phrase “punishing” users. For the user of the relevant package, it could be “punishing” to include a macro(?), not its absence (I guess the intention is though to always help the user, do you have a good counterexample?). I think Kristoffer meant punishing the user which is trying to register its package. Not doing it automatically might be taken that way, but I think it would happen anyway, at some point. [image] oxinabox: We don’t even block automerge for packages that have no tests. There’s a bit of a difference, then we are lowering the standard, allow doing less; we want more tests (which can be added later). We could require tests (or at least one…) maybe for 1.0 versions of a package? But doing a macro definition is doing more. if the macro is part of the public API, the function that the macro wraps should also be public (with public or export, in future Julia versions) and documented I’m confused about that. Why? The macro should be exported or public (if not, no change from status quo, since only for internal use), but does it follow from it that the function should be (which is just an implementation detail?)? I don’t use macros much (not defining my own), mostly e.g. @code_native and I never use code_native directly. It is exported for some reason though. Another potentially good restriction would be to whitelist macros that define new string literals. Why? It’s not very common to add such, and I guess could be discussed in each case. There’s a proposal for a new one, upper case S (for styled strings) into Julia’s Base, but it seems it will end up there, but not exported. [image] oxinabox: I mostly push back on excessive macro use by using this quote from @stevengj 's 2019 JuliaCon keynote. It’s a good quote, that I note most may not be aware of, and it seems an argument for not merging so fast. Since I don’t make my own macros, I neither do @generated functions (nor really taken the time to understand that concept and macro). Would that be something to restrict too (or rather)? I at least think the quote from @stevengj would apply to it equally, or even more.

[image] nsajko: A more conservative approach would perhaps be to only do this when the definition of the macro crosses some complexity threshold. This seems backwards. Unnecessary @eval and macros tend to simply tweak and paste the input into existing language structures. On the other hand, very useful macros can do very complicated code transformations (Accessors, BenchmarkTools, Tullio, LoopVectorization, etc), and their existence is worthy because it’s not straightforward to manually write out the @macroexpanded expression. And these packages tend to have restraint and only release a couple distinct macros and very similar variants. Why restrict them more than someone who forgot about parameters and higher order functions and designed 5 dozen macros that each do barely anything?

I feel like macros in Julia are a quite self-regulating because they are difficult to implement. At least every time I’ve written a non-trivial macro it quickly turned into “This is really hard… Is there any way I can do this without writing a macro?” As such, I don’t see that there is a problem with overuse of macros across the ecosystem. Also, in the other thread, there seems to have been some confusion between defining macros and using macros. Difficulties with, e.g., jumping to the definition of a macro are a tooling problem (and IMO, immature tooling may be Julia’s biggest concern, but just a consequence of the language being so young), not a problem of macros per se. Certainly, macros are very much at the core of the language (@test, @assert, @show, @warn, @printf, …) and calling them is completely normal. As for writing macros, the warning in the manual applies, but I really don’t think we need to do anything to nudge people away from them beyond that. Are these ideas good? Sorry, but no :wink:

[image] goerz: Certainly, macros are very much at the core of the language (@test, @assert, @show, @warn, @printf, …) and calling them is completely normal. Thats what the other thread is trying to convey: It might be obvious to you, but currently, there is not clear, explicit teaching, that calling macros is completely fine for everyone, and so, I think it can get confusing to people. Lets make that distinction clear, is all what I am saying. :slightly_smiling_face:

How to discourage macros: disable automerge of General registry PRs?

Internals & Design

ShalokShalom October 3, 2023, 8:45am 2

Me and nsajko launched our posts simultaneously, can somebody please merge them?

Topic		Replies	Views
Heavy macro use or not? Community	24	1459	June 5, 2024
General Registry: Create tag for macro usage New to Julia	6	599	October 3, 2023
Why distinguish macro names with @? Internals & Design	47	2433	May 14, 2022
How to warn new users away from metaprogramming Internals & Design question , proposal	27	7364	November 6, 2022
Mixin and parameter packages: How many macros is too many macros? Community question , announcement	10	1656	May 11, 2018

How to discourage macros: disable automerge of General registry PRs?

Related topics