The State of Julia for Scientific Machine Learning
A brief survey. Considers why Julia isn’t more popular vs. Python & Jax. Finds calling Julia from other languages difficult.
7 Likes
While calling Python, R, C, and C++ functions in Julia is easy with the PyCall.jl [54], RCall.jl [60], and the Julia standard library function ccall [61], the converse is often extremely difficult and unintuitive as you need to explicitly manage Julia’s context and the passing of complex datatypes between languages.
This is already outdated with PythonCall.jl existing (for quite long even), which is a bidirectional, and it is easy(er) for calling Python, and as easy to call from Python to Julia with it?! Something as good for calling from R may be needed, I mean a JuliaCall R package exists, already useful, but could be improved.
10 Likes
The key point of that paper is that Julia aims at taking over Python in ML, and that not achieving that means a failure for the language. We know that this is a false claim, but unfortunately the publication of texts like that only spread it over the Internet, increasing its appearance of truth.
12 Likes
I just skimmed a bit through, in my view it’s quite subjective:
It’s quite surprising to talk about scientific machine learning in Julia but not mention ChainRules.jl, DifferentiationInterface.jl, Enzyme.jl, …
In present year, Julia users have mostly moved away from improving the Julia language and are now
more focused on developing libraries for specific projects. With love, we argue the Julia community
has moved on too quickly. The state of machine learning is rapidly changing, and Julia has the
potential to address many of the pain points in the community.
Also: how is the Base language connected to state of machine learning? Neither Jax, PyTorch, etc. are base language Python features but instead independent libraries which appeared decades later. Development is much faster in packages than in the base language. Hence Julia tried to move out code from Base to independent packages. Same is true for Python.
Julia’s lack of software engineering-centric features, lackluster debugging infrastructure, subpar in-
dustry adoption, its “rivalry” with Jax, and insufficient interopt all bottleneck further adoption
I don’t really see how JAX is much better in those? JAX runs on windows with GPUs? JAX has a better debugger?
Overall, for me the paper does not really provide a comprehensive overview around machine learning and especially details about automatic differentiation are definitely lacking.
20 Likes
I think this is not even true, there is so many thing being improved in the core language still, like Memory types now and TTFX.
Ok, I did not move away from the core Julia language, I never started working on that, but well…
9 Likes
This is really the primary flaw of the paper. They frame all the shortcomings as a language-level problem rather than a package ecosystem problem, but they entirely cite third-party libraries for machine learning, testing, and static analysis as strengths for Python. This line…
While Python has a clear list of future goals [2], and individual ecosystems within Julia such as SciML have roadmaps [74], the Julia language itself has nothing but surface-level GitHub issues.
…was not only ignorant of base Julia’s openly discussed long-term developments, it just links Python’s PEP 0 index (Python Enhancement Proposals), which anyone can see for themselves are not much involved in key developments for machine learning. The dismissal of Github is also strange because draft PEPs are officially instructed to be submitted as pull requests on Github and automatically published on the main website. If Python didn’t predate Github, PEPs might just live there.
Besides the outdated Python interop and strange double standard for holding Julia solely accountable for interop from other languages, this doesn’t make sense to me because converting types between languages is difficult in either direction. Python’s dictionary-backed objects don’t magically turn into Julia structs or vice versa, so interop is often suboptimal even when it’s pulled off.
15 Likes
While this paper is a bit sloppy in terms of distinguishing the language from the ecosystem, I think it make a number of valid points.
- The Julia language itself is in general more useful than Python for numerical computation (let’s hope this is true because Python isn’t intended to be a numerical language, while this is Julia’s main intent with explicit effort to mitigate some of Python’s shortfalls).
- Julia’s ML libraries are less popular than Python’s but are competitive in terms of performance, and slightly more ergonomic.
- Julia’s software engineering infrastructure and tooling is not as extensive or easy to use as Python’s.
With that being said, I think this paper is a bit sloppy in its presentation in missing some vital distinctions.
- When competing against Python’s ecosystem, you’re not competing against Python as a language, but against the languages that Python’s ecosystem are written in (for example, C++).
- Because most of Python’s ML libraries are black-box, most types can be written as matrices. This means that you can have the system optimally compile (and autodiff) for only small set of types (which is what Jax does). Julia’s main advantage is being able to optimally compile for arbitrary types which is a much more challenging problem. However, solving this problem doesn’t really give Julia a strong advantage in black box ML.
- SciML is a form of explicit gray-box modelling, which means that in order to make the models maintainable and inspectable, object fields need human-readable. It also means that simulation tools (such as DAE solvers) need to be differentiable. Jax simply doesn’t support this, while Julia is pretty much the only ecosystem that can support this. Not only is Julia ready for this, no one else is even close.
- Static analysis requires static compilation, which the Julia community is working on. Once static compilation is complete (along with static analysis), Julia can compete with C++ et. al. as being a language to write Python libraries in, but this won’t compete with Python.
- The kinds of software engineering testing/tooling being commented on requires some formal notion of interfaces, which is really hard with multiple dispatch (MD). This problem has been solved for OOP but not MD because not only do data structures require interfaces, functions require them too. I heard that the Julia community was also working on this but I have no idea how they will be able to solve it. Correct me if I’m wrong, but the benefits of MD may come at some form of cost with respect to clean interfaces.
9 Likes
Quoting the paper: “… Julia’s language-level issues that are preventing further adoption.”
What a load of crock. There are no bona fide language-level issues even mentioned in the paper as problems. Libraries? Perhaps. Language? I think the authors misundertand what a “programming language” is.
8 Likes
Y’all, this looks like an undergraduate conference proceedings. Let’s not pile on too hard at them.
31 Likes
Totally agree.
On the other hand, this is accepted for the NeurIPS Machine Learning and the Physical Sciences Workshop (ML4PS), Perspectives Track.
It’s not a Neurips paper but still something internationally well known and the criticism here is fair and constructive.
But I heard from one of the authors that some of our points will be included in the camera ready version.
So I’m happy that this kind of open-review here has an impact on the final paper 
16 Likes
I think this applies equally to anybody who made an effort in good faith. There’s a world of difference between analysis and trolling, and the former deserves the same respect we’d give to any colleague, no matter what faults we identify. I certainly hope the authors received the bulk of the criticisms here as respectful feedback.
7 Likes
Maybe, but I used to be an academic and I can’t tell from reading the text what the authors’ highest educational achievement is (nor am I sure why it would matter as long as they are able to make a cogent argument). And the discussion here has been very fair to point out the problems in this paper, I don’t think this is in any way dogpiling on them (with the exception of perhaps one post).
Sure, it’s just easy to forget that there are real people behind some work, especially if they aren’t speaking up here. My post was aimed to be more preventative than scolding — just a nudge to keep the criticism constructive — but I can see it may have misfired a bit.
Knowing a bit about the authors is often helpful for me in evaluating academic papers and how to best respond.
16 Likes
It’s good to see this topic get attention in general. There are some issues with this preprint (I see the existance of Supposition.jl for PBT is missed, for instance), but also some rather valid comments such as:
Complex Debugging Messages
On a language level, Julia’s error messages have continually been an unaddressed pain point in the community. The complexity of multiple dispatch, as well as the notoriously long stack traces, make digesting Julia’s error messages a tedious process.
IMO, it’s helpful for us to hear about both perceived and real issues. Perceived issues can benefit from more PR/awareness campaigns of solutions (e.g. promoting Supposition.jl for testing), and the real issues deserve attention. For instance, by the end of this year I’m hoping to have draft PR up for a new framework for richly informative error messages (
: do get in contact with me if this interests you).
4 Likes
This has been on my list to respond to for a while.
I know Eddie and Jacob personally – we are part of the same department (Mathematics) and we usually see each other either passing through our lounge or at the weekly department tea time 
(sadly missing it today due to a conference).
I want to give praise that Eddie and Jacob both brought up good points within this critique (and continues to be so). As is discussed here, some points are readily addressable, but I think the core conceit of the paper:
We ask the Julia and scientific machine learning communities: what is the future for Julia?
Has been partially achieved in the discussion here on this discourse thread. I am excited to see what other outcomes will be from the Conference. Moreover,
This is exactly what I love to see when it comes to pre-prints and having discussions. The ability to have these community discussions to improve the overall concept/idea of a paper is fantastic and I think a strength of our community.
Yes and yes to @mbauman’s point. I think the Julia community is at its best when we focus on building bridges and building up one another – thoughtful critiques included. The community is comprised of wonderfully passionate people but I think it is also full of wonderfully compassionate people. Let’s continue supporting one another in that spirit.
To that end, Eddie and Jacob, if you see this and want to, please feel free to grab me the next time you see me in the Math Dept. lounge or at our weekly tea times. I am happy to chat more about these points raised in the paper as well as help with any editing or to give some feedback as a reader.
Cheers and keep up the great work!
~ tcp 
P.S. This is Jacob S. Zelko (email: jacobszelko@gmail.com)
14 Likes
Since I may be looking at Machine Learning myself in the near future, in the context of quantum control, I found it interesting that according to this preprint, the limitations of Julia in the context of machine learning aren’t at all specific to machine learning. These are the same pain points (“lack of tooling”) that Julia has in general, and which I attribute mainly to its relative youth. I very well remember the state of Python’s tooling in the early 2000’s (when Python was about as mature as Julia is now), and it wasn’t anywhere near what it is today.
Since I’m already familiar with these limitations, I don’t think they would really impact whether I would choose Python or Julia to reproduce examples from machine-learning-in-quantum-control in the literature, or prototype my own solutions. What would affect my decision if particular ML techniques were not available in Julia libraries, i.e., feature parity between let’s say keras and Flux. Or, if the backends for training neural nets were at significantly higher performance when called from Python vs any pure-Julia solution.
The preprint doesn’t really mention any of these concerns, which actually makes me pretty hopeful that I’m not going to have too hard of a time getting familiar with the relevant ML techniques entirely within the Julia ecosystem, and that, e.g., the outcome of adding RL to QuantumControl.jl would end up competitive with what others in the field have done in PyTorch or Jax.
3 Likes
Jax has also released a few very broken releases over the last few weeks making this point laughable
2 posts were split to a new topic: What are the future plans for Scientific ML?
Does anyone have a genuine usecase for calling Julia from another language?
What are the use cases for calling Python from another language? I can only think of 1. You are writing an application and you want to embed Python into it to provide users of your application with a Python UX.
It’s a pretty niche usecase, if for no other reason that there’s a bunch of potential security related issues letting users of your application inject arbitrary executable code into it.
I wouldn’t pay too much attention to this paper. It looks like somebody felt like they needed to publish something to get their number of academic papers higher for the year end.
I wouldn’t pay too much attention to this paper. It looks like somebody felt like they needed to publish something to get their number of academic papers higher for the year end.
I doubt this is the case given the credentials, but as already asked for in arXiv: "The State of Julia for Scientific Machine Learning" by Berman & Ginesin - #9 by mbauman it would be best to remain constructive.
4 Likes