next week I am once again giving the Julia Zero2Hero workshop at the University of Exeter (now updated to Julia 1.9, including a Makie tutorial!).
As I am advertising Julia more and more, I constantly have the same question being asked at me “why Julia?”. Originally, I had information in the aforementioned workshop, but that information was becoming more and more extensive.
I’ve decided to split it off into a README repo, whose purpose is to have a well-organized, well-formulated answer to “Why Julia” (in particular in the academic context).
It would be great if you have a look and correct me in places that you think I say incorrect statements, or perhaps I am claiming too much. While this post reflects my opinions, I want it to be as factual as possible nevertheless.
Furthermore, feel free to contribute more detailed advantages of Julia in the last section of the document (the detailed bullet point list), assuming that you share my views on Julia
Very cool, I actually need something like this to explain why Julia is worth it at a meeting
This is something I think it could be improved:
For 5th section:
Julia is ahead of Pytorch
Pytorch is very advanced and on Discourse there are a lot of discussions where people advice to still use Pytorch for traditional workflows. So I wouldn’t say so. Maybe more that it is keeping up with it for traditional workflows and has advantages when you want to do something custom like writing a kernel yourself. See Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet?
For 6th section:
Julia is written in Julia
This is more of a title issue than anything else since on the text you don’t mention that the compiler is written in itself, but the title hints at this from a user perspective. Consider that most of Julia is written in Julia, but there is a lot of C and other languages in the main repository. To be honest, also CPython is written mostly in CPython if you look at the composition of programming languages used in it. But surely Julia has standard libraries + Base written entirely in Julia, which is cool. See How is Julia written in Julia?
I would quibble with some of the claims in the repo. In particular
I think maintainability (debuggability, static analysis, stability, tooling, ease of refactoring) is another major core aspect of a programming language - arguably more important than ease of writing. You only write a program once, then you maintain it for years.
I would stress that the “two language problem” refers specifically to low vs high throughput languages. If your problem with Python is not its speed but its latency, then Julia will do not better, and probably worse. If your problem with Python is problems with maintenance, then Julia will do no better, either.
I would caviat the claim that Julia is written in Julia. Much of the compiler, the threading runtime, the GC, symbols and strings are written in C/C++.
I would not say that Julia’s package ecosystem is top-of-the-class in many scientific fields. Maybe a handful? I would bet that the ecosystems where Python outshines Julia outnumber the opposite by at least 10:1.
Thanks a lot for the feedback Jakob, that’s the kind of discussions I need!
I agree that maintainability is important, but I still stand that in academic usage, the other 4 reasons I list are more important. Speed of writing is by far the most important in my academic experience. The majority of scientists do no need to develop a software library to do their jobs. Thus, I would argue that “You only write a program once, then you maintain it for years.” is not the common case in academia. The common case is that indeed you write a script once to produce a figure for one paper and then you may likely not reuse it again.
That is correct, latency is an issue, but it appears unrelated to the two language problem. I don’t see any obvious way they relate. I believe that in the academic circles the “two language problem” is well established to refer to what you also state, hence I think it is not necessary to explain it further in the document.
100% right, user Tortar raised the same issue, and I corrected the repo accordingly!
“I would bet that the ecosystems where Python outshines Julia outnumber the opposite by at least 10:1.” I would need data to believe that actually. I would believe it blindly 2-3 years ago, today not so sure. But, sure, phrasing it more diplomatically is a good suggestion. I’ve changed it to “some”.
To me, one important thing is the quality of packages. Examples:
As late as 2010, the Optimization TB in MATLAB gave a wrong/very inaccurate solution to unconstrained QP problems. I got much more accurate solution by implementing the Lagrange multiplier method myself (which is trivial). Of course, this particular problem has been solved by now in the O TB, and I have no reason to doubt the quality of commercial TBs from MathWorks today, but quality is important.
When I tried to learn Python in 2011-12, one package I tested was an optimization package written in pure Python. The quality was less than impressive… to find the optimal solution to the (2D) Rosenbrock banana function (a standard test case), I had to give an initial guess that was within 10% from the optimal solution. Of course, linked in Fortran/C code in SciPy were much better. And I’m sure a lot of Python code is of qood quality. Still, quality is important.
My impression is that a large number of Julia packages are of very good numeric quality. To me, that is important. I also like (i) the syntax of the language, and (ii) that many developers are quite young, leading to a dynamic development environment with a certain level of future.
Multiple and dynamic dispatch complement each other, but they are quite different things. Dynamic dispatch leads to choosing the right method at runtime, so we can implement predicate dispatch or combine code with abstract (non-inferred) and concrete (inferred) types. It is based on type inference, method specialization, method call inlining:
Metaprogramming: Generated functions - we can use type information at parse time and generate specialized code Abstraction in technical computing pages 78-79
We can leverage compiler complexity and solve problems just adding new syntax or macros that solves our problem before compiler stage, e.g. Loop fusion: More Dots: Syntactic Loop Fusion in Julia
4. Composability
I would mention a bunch of interesting solutions in platform and package manager like:
Persistent binary artifact storage + Yggdrasil + build system for any platform combinations: Home · BinaryBuilder.jl
Clang.jl - automatic C wrapper code generation (used in BinaryBuilder) that helps for many third-party libraries to just work
My view of the two language problem is much more pedestrian. I never prototyped things in python to later translate to Fortran. When writing numerical code Fortran was good enough from start for me.
But then I had to distribute my code, even to my students, which would want to produce plots, data frames, statistics, etc. For that the solution was a patchwork of bash, python, TCL, web interfaces, etc. The best possible scenario was when I wrote a python interface to the code. And from there to actually have a standard distributed package there was still a barrier which I never overcame.
With Julia not only I write the numerical code, but make it into a package, write the docs, thus making many original throw away scripts useful tools for my group and, hopefully, to other people. And the students are able to use the packages and do all further analysis (non critical in terms of performance) within Julia. I am even using Julia as an interface to distribute cross platform binaries of packages.
I do have now a commitment with maintaining packages for long, but that’s a good thing, for my research group and for the impact of my work.
I’d add that while writing Julia code with the highest possible performance is not super easy or clean anymore, people can package these optimized pieces of code up into much smaller libraries which others can then plug into their own code to benefit from that. For example in numpy, you opt into the whole ecosystem or not, you don’t really pick an algorithm here or there because it all assumes working with the framework. So it’s nice that their algorithms are also tuned to highest performance, but they are not so highly composable. In Julia, that’s different. Usually, only the scalar kernel functions have to be tuned, the rest can be orchestrated with broadcasting or simple map calls in most cases. I think that lowers the barrier for contributing.
I agree a lot with this in general, but want to add a caveat. In Python, interop is better for the biggest APIs and worse for the rest. (I think a lot of this is because Python’s interfaces are a very recent feature, with the limited adoption that entails.) ~99% of all code will work if you pass a Pandas DataFrame through it, or anything else that implements the Pandas API. On the other hand, almost nothing works with Polars (which explicitly chose not to follow the Pandas API because it leads to slow, difficult-to-read code).
With Julia, OTOH, if you try passing something through a function (like a named array), there’s an 80% chance it will work. This is true regardless of the implementation you pick, because Julia has built-in (unenforced) “interfaces” that are widely used (e.g. the array interface); but because they’re undocumented and unenforced, there’s always a very good chance your package just won’t work with it.
It’s not as fundamental as the other points (and it might just be personal preference), but in my continuing Julia learning journey, I found that transitioning from Jupyter to Pluto helped greatly; it’s where I go to work things out.
