Motivation:
Interpretation: The ML space is large enough for Julia to have its place. Combining an excellent-grade ML ecosystem with Julia’s niche several things can probably be made.
But then, what’s the issue with the current Julia ML ecosystem? Let’s seek an explanation other than that Julia as a language is cursed never to become popular. (If that’s the case, there’s nothing to be done, nothing, other than dread and disappointment.)
Julia’s ML.
Machine learning is based mainly on two parts, Tensor manipulation and autodiff. You need a good compiler in both to do well.
So, what’s the issue? I think the problem is that Julia as a language is fast-changing, and compiler extensions are often forced to rely on undocumented, ever-changing internal. That means even though Julia tries very hard not to break things, these compiler extensions could easily break, erasing the work.
So, okay, but what can be done? Aren’t custom compiler passes and compiler extensions inherently dependent on compiler internal? Well, if we view the compiler as a monolith, yes.
However, the good news is that the compiler is not a monolith. It’s comprised of so many passes. Now, what we need to do is simply document these passes, and while some optimizations may be changed, the semantics of each intermediate representation and each pass could remain right?
But then comes more issues.
- The uncontrollable LLVM: LLVM itself changes and with it the Julia compiler.
- Some passes/IR may get deleted entirely. For example, if you used to go from A->B->C->F, you might replace it with A->D->E->F. If your compiler extension operates on B->C IR, then you’re doomed.
Solution: What if we stop viewing the Julia compiler as a single, monolithic thing and instead as a set of mosaic assets to be combined to make a “compiler for Julia”? As for the LLVM, the LLVM itself is also not a monolithic thing and has lots of options. Many LLVM extensions are made so that’s issues at the LLVM level.
So, now what? The answer is that my puny mind can only comprehend so much, and there will be issues upon issues with this.
But at least, theoretically, we could see the light at the end of the tunnel.
So, my question is when… Is it on the roadmap? Or is there any issue that will prevent it from ever being possible?
What to take away from this? Well, at this point, I trust in the Julia team to get miracles done somehow.