An update, since apparently some people are gatekeeping Julia adoption on the completion of these compiler work items: these items are all complete as envisioned at the time this was posted (and have been for a while). Additional details follow.
This is always ongoing, of course, but the compiler is very well debugged — the unreachable is quite hard to reach these days.
- Multithreading
This was already done at the time I posted this. As of Julia 1.7, we even have the ability for non-sticky tasks to migrate between OS threads.
Yep, done and merged in Julia 1.3.
Yes, that was also done.
- other thread safety
This was already pretty good in 1.3 and now in 1.7 we can say that it’s quite done. Moreover, the ecosystem has largely adjusted to the presence of multithreading as well, which is not a compiler work issue, but was very important. Multithreading is a very real and present thing in Julia and its packages now. It is widely and effectively used.
- Compile-time latency, aka “the time-to-first-plot problem”
- making compilation faster
This is done. The compiler got about and order of magnitude faster between Julia 1.0 and Julia 1.6. Depending on the speed of your system, time to first plot is now about 5-10 seconds, where it was originally 30 seconds to a minute or more.
This doesn’t mean that latency is solved because people are greedy and they don’t want to want to wait at all for their first plot. But it does mean that working on making the fully optimizing compiler faster is complete and it’s time to work on other approaches.
- caching more things
The biggest change here is that precompilation is now done at package install time and in parallel using multiple threads. This helps a lot with the perception of latency when using packages. You can also generate custom sysimgs to eliminate all startup time, but more work needs to be done to make that tooling better and possibly generate sysimgs for each environment when Pkg does operations on the package versions in it. This is the least complete work item as more exploration needs to be done here.
- Compiler-related packages and tools
Works quite well. The main problems at this point are that packages often do things that aren’t relocatable, which breaks this. But that’s an ecosystem issue, not a compiler issue. The tools exist to not do this — Artifacts, Scratch spaces, Preferences, DataSets — but packages need to use them.
- Debugger
Works and is pretty smooth and featureful. Sometimes too slow because it’s interpreter-based but that was what was predicted by everyone involved. History: the previous debugger was fast but very complex and therefore flaky. There was a lot of pushback to have something simple and reliable — this is that simple, reliable debugger. Now we see why the earlier debugger took the approach it did.
Debugger has a compiled mode where it can run code fast but breakpoints don’t work. There’s probably something clever that can be done here to improve the perceived speed, but there’s no ongoing work on it and there won’t be unless someone takes it up as a project or decides to fund work on it.
There are a number of other debugging tools that have different approaches and properties, like Infiltrator, which lets you put a breakpoint in your code by using the @infiltrate macro at the point you want to stop.
CxxWrap has become the de facto way to wrap C++ code. Cxx was great but it’s a very complex approach that is highly intertwined with Julia’s internals and the author has not had time to maintain it. CxxWrap doesn’t have the fancy C++ REPL feature, but it’s straight forward and reliable and doesn’t depend on Julia internals to work. If someone does want to take on maintenance of Cxx, I’m sure that @keno would be happy to help get someone going with that — it’s a very cool project that would surely give someone a much deeper understanding of both C++ and Julia.
- type checking/linting
The Julia VS Code extension has long included a good linter and JET does actual type checking of Julia code and more. Various independent parties have observed that JET seems much more capable and usable than similar tools in other dynamic languages due both to Julia’s type system and @aviatesk’s excellent work on the package. He and other members of the compiler team will continue to work on this functionality but it’s already state of the art.