How to discourage macros: disable automerge of General registry PRs?

I mostly push back on excessive macro use by using this quote from
@stevengj 's 2019 JuliaCon keynote.

Functions are mostly good enough for Jeff Bezanson.
Don’t try to outsmart Jeff Bezanson.

Video: https://youtu.be/mSgXWpvQEHE?t=578

20 Likes

:-1:

6 Likes

blocking automerge is a very heavy handed approach to influencing the ecosystem and creates a lot of manual work for registry maintainers. I think it should be used very sparingly, and something like defining a macro is a totally legit thing for a package to do.

IMO if you want to influence folks you need to convince them, e.g. write blog posts showing how some particular code that uses macros would be better without it (or PR such code). But your case will need to be solid to actually change someone’s mind.

22 Likes

To me this seems plausible, at least to mark the registration PR with a macro label and/or extend the the auto-merging period. [I’ve never second-guess anyone on adding a macro, also since I might not know, I have blocked some package registrations for other reasons, and occasionally look at code before registered.]

I’m not sure I agree with the phrase “punishing” users. For the user of the relevant package, it could be “punishing” to include a macro(?), not its absence (I guess the intention is though to always help the user, do you have a good counterexample?). I think Kristoffer meant punishing the user which is trying to register its package. Not doing it automatically might be taken that way, but I think it would happen anyway, at some point.

There’s a bit of a difference, then we are lowering the standard, allow doing less; we want more tests (which can be added later). We could require tests (or at least one…) maybe for 1.0 versions of a package?

But doing a macro definition is doing more.

  1. if the macro is part of the public API, the function that the macro wraps should also be public (with public or export, in future Julia versions) and documented

I’m confused about that. Why? The macro should be exported or public (if not, no change from status quo, since only for internal use), but does it follow from it that the function should be (which is just an implementation detail?)? I don’t use macros much (not defining my own), mostly e.g. @code_native and I never use code_native directly. It is exported for some reason though.

Another potentially good restriction would be to whitelist macros that define new string literals.

Why? It’s not very common to add such, and I guess could be discussed in each case. There’s a proposal for a new one, upper case S (for styled strings) into Julia’s Base, but it seems it will end up there, but not exported.

It’s a good quote, that I note most may not be aware of, and it seems an argument for not merging so fast.

Since I don’t make my own macros, I neither do @generated functions (nor really taken the time to understand that concept and macro). Would that be something to restrict too (or rather)? I at least think the quote from @stevengj would apply to it equally, or even more.

This seems backwards. Unnecessary @eval and macros tend to simply tweak and paste the input into existing language structures. On the other hand, very useful macros can do very complicated code transformations (Accessors, BenchmarkTools, Tullio, LoopVectorization, etc), and their existence is worthy because it’s not straightforward to manually write out the @macroexpanded expression. And these packages tend to have restraint and only release a couple distinct macros and very similar variants. Why restrict them more than someone who forgot about parameters and higher order functions and designed 5 dozen macros that each do barely anything?

3 Likes

I feel like macros in Julia are a quite self-regulating because they are difficult to implement. At least every time I’ve written a non-trivial macro it quickly turned into “This is really hard… Is there any way I can do this without writing a macro?”

As such, I don’t see that there is a problem with overuse of macros across the ecosystem.

Also, in the other thread, there seems to have been some confusion between defining macros and using macros. Difficulties with, e.g., jumping to the definition of a macro are a tooling problem (and IMO, immature tooling may be Julia’s biggest concern, but just a consequence of the language being so young), not a problem of macros per se. Certainly, macros are very much at the core of the language (@test, @assert, @show, @warn, @printf, …) and calling them is completely normal.

As for writing macros, the warning in the manual applies, but I really don’t think we need to do anything to nudge people away from them beyond that.

Are these ideas good?

Sorry, but no :wink:

13 Likes

Thats what the other thread is trying to convey: It might be obvious to you, but currently, there is not clear, explicit teaching, that calling macros is completely fine for everyone, and so, I think it can get confusing to people.

Lets make that distinction clear, is all what I am saying. :slightly_smiling_face:

I’d ask a rhetorical question: who uses goto’s? No one. Because it’s (now) common knowledge to avoid spaghetti code (except in some rare cases when you absolutely know what you do). Same for “unsafe” in rust.

My point being (not an original opinion here ^^), maybe a stronger statement could be added to the documentation.

But at least, this conversation is participating in “raising the awareness”, which is always a good thing :slight_smile:

3 Likes

I reject both premises. namely, that

  1. nobody uses @goto (they do!)
  2. it necessarily leads to spaghetti code (not always!)

yes you have to be careful. but Julia @goto only works to jump to a @label inside the same function anyway, so there is significantly less potential for abuse than classic goto

4 Likes

I actually used a @goto once. If I recall correctly, it was mainly to avoid a really long if block (and reduce indentation levels). Perhaps there was a better way, but I couldn’t think of one. The code for that function had lots of complicated exception handling, and I couldn’t figure out a more modular way to write it.

2 Likes

Using goto is the clearest way of implementing state machines.

4 Likes

If we get really nitpicky, early returns, continues, breaks, and exceptions are also one-way gotos, and some people e.g. Bertrand Meyer have considered them just as bad.

if we get nitpicky again, all the structures we use are just the maintainable goto patterns, you can see them in the compiled code. So if we don’t neglect the provided structures, carefully using a @goto to do something those structures can’t (like a multi-level break) isn’t abnormal. A lot of really bad GOTO practices are prevented by @goto not being able to jump into or out of function bodies. However, it can jump between let/if/begin blocks and out of a for block (the REPL just dies if you try to jump into a for block), so the docstring saying “can’t jump between top-level statements” doesn’t describe the limitations that well.

3 Likes

This is actually pretty nice. I don’t think I ever used it in Julia, but it can be useful when one wants to exit multiple nested loops at the same time. A simple break only exits the innermost loop.

Some languages actually have labelled breaks, that enable using break for this purpose (breaking out of multiple nested loops) instead of goto. Go is an example as far as I remember.

4 Likes

It’s the go-to (different meaning, no pun intended) example of a clear and useful GOTO that typical language’s structures don’t provide. I’m honestly surprised that break and continue statements don’t typically come with an optional label (break _label_) or depth (break 2) restricted to the 1 active nested for-loop (so a for ... function ... for ... is not regarded as a nested for loop because the function’s inner loop does not run by default in the outer loop).

2 Likes

gotos are not a very good analogy for macros. Instead, a better analogy would be functions (since that’s really all macros are, they’re functions that rewrite code). All the usual complaints people make about macros could be made about functions.

Maybe we should have a registry flag that discourages people from writing functions in their packages since it’s so confusing and functions can lead to spaghetti code where definitions for things are all over the place and obscures what’s going on from a user!

2 Likes

Well, in the end everything becomes a goto (or function call) sooner or later:

julia> function f(n)
           for i = 1:n
               println(string(i))
           end
       end
f (generic function with 1 method)

julia> @code_lowered f(2)
CodeInfo(
1 ─ %1  = 1:n
│         @_3 = Base.iterate(%1)
│   %3  = @_3 === nothing
│   %4  = Base.not_int(%3)
└──       goto #4 if not %4
2 ┄ %6  = @_3
│         i = Core.getfield(%6, 1)
│   %8  = Core.getfield(%6, 2)
│   %9  = Main.string(i)
│         Main.println(%9)
│         @_3 = Base.iterate(%1, %8)
│   %12 = @_3 === nothing
│   %13 = Base.not_int(%12)
└──       goto #4 if not %13
3 ─       goto #2
4 ┄       return nothing
)

Thus, for gets rewritten by the compiler – according to the docs into an equivalent while loop calling into the interface methods for iteration and in turn, into goto’s as the lowered code shows.

To me, macros are somewhat like user-defined special forms, i.e., language keywords with special semantics (slightly restricted to local code transformations though). On the one hand, they should be used sparingly as they extend the language with new keywords [1]. On the other hand, there can be good reasons to extend the language, e.g., documenting definitions or simplifying notations in special domains [2].


  1. Clojure has (with-open [f my-file] ...) for file handling, whereas the do-notation in Julia achieves much the same via open(myfile) do f .... Yet, do is just another build-in macro. ↩︎

  2. Good examples for definitions are @testset, @test and Tullio.jl is just awesome. ↩︎

I don’t think this post really thought through the implications. Yes, macros can be abused. But what are the packages we’d be omitting here:

  1. Packages that implement domain-specific languages (JuMP.jl, ModelingToolkit.jl, Turing.jl, Catalyst.jl, etc.)
  2. Packages that implement performance optimizations via macros (LoopVectorization.jl, MuladdMacro.jl, etc.)
  3. Packages that use macros as small syntax sugar for users (Symbolics.jl, Tullio.jl, etc.)

If you couldn’t tell from the list, this rule would be omitting almost all of the most widely used packages in the Julia registry? Are we really going to basically remove the automatic merge for every major package because it’s possible to abuse macros?

I mean, yes I agree that you can misuse macros, but it’s very heavy-handed to then make almost every major package no longer get updates without manual intervention.

4 Likes
macro is_this_a_good_idea(x = "")
	"no"
end
1 Like

I’ll admit I failed to account for useful packages like LoopVectorization.jl while writing this post, however:

  1. Judging by GitHub - JuliaSIMD/LoopModels: "Full speed or nothing." - James Hetfield, and some of the warnings and limitations of LoopVectorization, the current implementation of LoopVectorization seems to be a limiting factor. So I think the macro-based approach may be a hack after all, despite how useful it was up to now?
  2. Symbolics.jl was actually one of the packages that prompted this post. Notice that I wrote “if the macro is part of the public API, the function that the macro wraps should also be public” in the OP above. Symbolics offers the @variables macro which is useful for interactive use, but there is no public API for using similar functionality as appropriate from noninteractive (package) code! There’s an internal function, Symbolics.variables, but even it only offers a semblance of the functionality of @variables, to quote its documentation:

Use @variables instead to create symbolic array variables (as opposed to array of variables).

One relevant issue:

Agreed. Also, a common trend in this discourse is that we forget the diversity of programmers wandering in here. Academic codes, domain-specific codes, scripting codes, production codes, package codes, etc.

Hence the: “Use it only if you reeeeeally know what you’re doing”. And consequently, status quo is fine.

Btw … hence the comparison I suggested about goto’s
And judging by some responses, I’d respectfully say that my point was quite misunderstood, which was not about the internals of goto - especially at assembly level !! :grimacing: - but more about its usage (which is arguably quite seldom … and should I remind that rare ≠ never ?).

Edit: typo and clarity

1 Like