You might want to track the progress on ONNX.jl - when it’s ready, you should be able to load any of the thousands of the pretrained models to Julia (e.g. see instructions for HuggingFace models) as well as run Julia models on mobile. We are not there yet, but some simple examples like Resnet are already functional.
One obstacle though, and, in my opinion, the biggest challenge for Julia, is that it provides an open and extensible ecosystem for machine learning. For example, JAX has ~200 operators, ONNX - ~250, PyTorch - around 2000. Every operator there is guaranteed to support backpropagation, work on GPU/TPU, have good documentation, etc. Sometimes it’s hard to support all of them, yet possible.
In Julia, we essentially have an infinite list of operators scattered over multiple packages. There are no formal requirements for adding new functions, and authors usually only cover their own needs. For instance, someone adding a new
ChainRules.rrule() may not check if it works with
CuArrays, somebody creating a new closure-based layer may not think about exporting it to ONNX, etc.
For a couple of days I’ve been thinking of a curated list of high-quality operators and language features with particular guarantees, e.g.:
- support autodiff (e.g. via
- support GPU
- support low-precision element types (Float32, Float16)
- have docstrings & be discoverable (e.g. be listed in some popular document)
- have performance tests
- support import/export to/from ONNX, XLA, etc.
We can then create automatic tests to ensure the quality of the listed features. Tests themselves don’t have to be exhaustive - these things must be tested in their packages anyway - instead, they will show the level of maturity of the ecosystem and highlight potential issues.