What was the motivation behind Julia's staging (specialization, macro, generated function, etc.) systems design?

Internals & Design
1
Background (what I learned trying to make a language even though it didn't work out due to unforeseen difficulty my naive mind didn't see coming.)

I’m back to Julia for a bit. It is still a language that does wonders anyway, even though I realize it has cursed edges.

I even thought that programming languages were simpler than they actually are, and wondered why some features like XYZ were not implemented. In some ways, it could be, but it is much easier to make a programming language that compiles your code than to make a robust language that robustly compiles the code of programmers who haven’t read through the whole compiler. Julia does a decent job, as much as I run into the rough edges of the programs.

But now, I started to understand the important thing behind fast dynamic language, staging. If you could stage the interpreter, you might be able to jit-specialize it to run as if it were compiled, for example. So, without further ado, let’s proceed to the question.

It took me like a year to clarify what is intuitively wrong with Julia, but I want to understand why it was designed this way.

So, the key to performance in a dynamic language is staging. Runtime code generation, dynamic compilation, and so on are normally slow unless you stage them carefully so you don’t end up generating/interpreting code in the hot loop. Julia does make use of a lot of these.

However, Julia has AT LEAST 3 staging mechanisms, and they don’t always interact with one another really well.

  • Macro expansion: happens at syntax time only. Sees no argument types.
  • Type inference: happens when a function with known argument types is put in. If an unknown argument type is given, the function returns to dynamic dispatch.
  • Generated functions: Functions with implementations generated during compilation. This is type-level metaprogramming, but AFAIK, the method table is frozen at generation time, and the world age doesn’t advance.
  • Evals: Dynamically evaluate Julia code at the top level, but the world age is only updated once the function reaches the top level; it does happen if you don’t wrap everything in a function or if the evals were called in a setup before “main”.
  • Packages: Some packages probably add their own ways of staged evaluation.

Now, each of these has its own distinct staged information, invocation time, invalidation rule, and generated precomputed/compiled result. They don’t always interact well with one another. (For example, macros can’t see the argument types; specializations can only dispatch by types, etc.)

Now, staging (and staged compilation) is important in fast dynamic languages, but for that to be used well, I think you need a way to manage the stages and not introduce many disparate staging mechanisms that overlap, but with each having a vastly different rule. That would be managing many systems at once.

But well, for some reason, despite me flagging this as a flaw, Julia still works well for many tasks, so maybe there is a reason I couldn’t see.

Why does Julia need so many distinct staging systems?

2

As you yourself said, each occurs at a different time and has different information (as well as a different level of abstraction) and hence can do different things well.

It’s like asking why a camera system has so many components: lenses, sensors, image synthesis from sensor readings, compression, maybe even post-processing like inference and reconstruction tasks.

I’m not going to go into specifics here, because you didn’t ask a very specific question. Loads have been written on what macros are (and aren’t) good for (e.g. domain-specific syntax), on what generated functions are good for (e.g. low-level type-based high-performance code generation), what dispatch is good for (e.g. solving the expression problem), the cases where you want runtime eval (e.g. interactive coding), and so on.

(As we’ve often commented to you, extremely broad and unfocused topics are discouraged here because they usually lead to uninformative, time-wasting threads.)

3

Perhaps I was hallucinating that staging was a monolithic concept you could combine into a single system.

4

I think it probably would be possible for the interfaces to our various staged programming look more similar to each other had we designed them all at once, but it’s probably too late for that now

5

There have been languages and language extensions that have a more rigorous approach to (certain parts of) staged programming like Scala LMS, but they have largely been academic affairs. There is a hierarchy of information available at different stages (syntax, partial types, types, constprop, full traces etc) and it would surely be nice if it was a bit more complete and “programmable” in Julia for lack of a better word.

Some people like compiler authors have struggled with Julia’s staging setup, and it clearly has warts that the community tries to fix with things like RuntimeGeneratedFunctions.jl and similar.

But in the end macros, generated functions, and compiler plugins (such as they are) cover enough of the space and were developed at different times for different purposes.

6

Okay, maybe I wasn’t fully hallucinating after all. I know there are some disagreements, but I think the current answer seems to be that various staged programming systems in Julia were developed at various times for various purposes that they look like disparate systems bolted on top of one another rather than a more coherent system people wanted. Thank you.

7

I think your summary is quite uncharitable. Macros, specialization and generated functions really do address different problems, and you have to strain to see them all as a coherent system:

  1. Macros take syntax and generate syntax like macros in most languages.
  2. Specialization allows you to choose different implementations based on type
  3. Generated functions allow you to generate largely arbitrary code based off types. If you really squint it’s a hybrid of 1 and 2
  4. Eval, the fully dynamic do anything to generate code function and the problem child for which the first 3 exist to avoid.

The first two are the real meat and bones of Julia, and unless you’re really sure you need it that’s what is most supported because 3 and 4 are so flexible they make inference and compilation quite difficult. A language that has a more smooth/complete/extensible hierarchy of binding time tools could probably exist but it’s not clearly feasible or necessary.

8

Okay. My summary might’ve been too harsh. I acknowledge the engineering limitation (Julia was written by humans after all) and that designing various staging mechanisms to be more “coherent” would be difficult.

And I would admit that Julia is pretty good at solving a lot of problems even though there are some rough edges that force workarounds.

I consider Julia as a language highly, actually speaking. The problem is that I often pushed my system to the breaking point, sometimes unintentionally, making it look flawed some of the times.