What does “moving compiler out of Base” actually entail?

Internals & Design
1

The compiler is being moved out of base. What does this entail? Does that mean that the compiler extension/custom compiler pass/etc will soon no longer rely on undocumented internals? If so, this could prove very beneficial to the automatic differentiation/etc ecosystem that rely on these features. However, I don’t really know enough what this actually means for the Julia ecosystem to know if this was the case. Can anyone explain to me what this actually meant?

Edit: It seems it no longer is about moving the compiler from core into base, but rather making the compiler a separate package. It’s so fast-moving at this point.

1 Like
2

Can you point on the broaded context, i.e. where you have read that expression?

3

This is in reference to Make Compiler an independent package (that lives in the julia repo) by Keno · Pull Request #56409 · JuliaLang/julia · GitHub (and followups). The TLDR is that there are 2 goals of this work.

  1. Up until ~2 weeks ago, the Compiler and Base had completely separate imports of a number of the core files (e.g. array.jl), which predates the existence of world ages. This meant that Compiler.getindex was a different function than Base.getindex which made working on the compiler a lot more annoying than necessary. Now, the compiler uses the “normal” Base methods (but at a frozen world-age so you don’t invalidate the compiler.
  2. Up until 1 week ago, any external package that wanted to use the internals of the Compiler was using the same Compiler module that Base uses to generate code, which meant that keeping those packages working on the development version of Julia required constant maintenance. Now, Complier is a semi-standard-lib package, so packages that rely on the compiler, can rely on a specific versioned package that follows normal SemVer rules. This means that when you update your Julia version from today’s nightly to next week’s nightly, the version of Compiler that packages like GPUCompiler and Cthulhu will see will be one that they know how to deal with.
14 Likes
4

FYI it should be “moving compiler out of Base” not “into”. I edited the title to be less confusing to other people.

In a similar spirit, many things have been moved “out of Base” in the past couple of version notably REPL and LinearAlgebra.

2 Likes
5

Both are true. It moved out of Core.Compiler into Base, and then (semantically, but not physically) moved out of Base into its own top level package. We just did both at the same time to avoid breaking downstream packages twice.

10 Likes