See Frequently Asked Questions · The Julia Language
Julia’s performance advantage derives almost entirely from its front-end: its language semantics allow a well-written Julia program to give more opportunities to the compiler to generate efficient code and memory layouts. If you tried to compile Matlab or Python code to Julia, our compiler would be limited by the semantics of Matlab or Python to producing code no better than that of existing compilers for those languages (and probably worse).
“Walks like python, runs like C” is just a catchier phrase than “Can look like python under certain circumstances, can run like C under certain circumstances, can sometimes even look like python and run like C at the same time, although maximum performance usually needs more verbose code”. But that’s still a good value proposition.
Like most programming languages, Julia could have a catchy slogan without an explicit comparison to other programming languages. For example, Rust’s slogan is “a language empowering everyone to build reliable and efficient software” and Python’s is “a programming language that lets you work quickly and integrate systems more effectively”. A slogan for Julia could focus on its growing importance in science and HPC.
I don’t think Julia looks like Python much, even the most contrived examples omitting obvious differences, like classes vs structs, don’t look the same. “Walks like Python, runs like C” is just a catchphrase version of “a dynamically typed language that can be written to let a compiler infer types and optimize like a statically typed language”. For what it’s worth, that phrase doesn’t appear in official sources; the documentation’s first page outright says performant Julia is not like interpreted Python or R. I think the myth of optimized Python persists out of wishful thinking and is exploited by clickbait that distracts you from the fact that the reference implementation CPython has every incentive to pull this off but hasn’t. Even the aspirational Python superset Mojo doesn’t optimize Python code.
My 2cts from personal experience coding in C/C++ a million years ago, in Python and then in Julia.
I was never good enough at C/C++ that I could write big programs fast, although I did like the ecosystem, workflow and speed.
Writing Python, I would always stumble upon a speed problem, where I just couldn’t get any improvement easily for say hot loops of custom code. Extensions of the language solved some problems locally but felt more like patches here and there.
In Julia I love that I can get to my desired output (ie implementation of functionalities) with minimal sweat. And if performance is not satisfactory, in can rethink bits of code here and there within the same language, by just being a bit more smart about how I do things. And without having to use some hacky stuff (or stuff that felt hacky) like in Python. So I think the 1.5 language is a bit misplaced, in the sense that everybody can write poor code in any language, but at least Julia allows you to get pretty close to C performance without massive re engineering (and dare I say with minimal effort) which is a huge benefit for what I do.
I never understood “looks like python” to mean exactly like python, more like simple syntax without too much boilerplate
Can someone tell what is a “static Julia” status at this moment and its prospect for future? I don’t dare ask Julia team for more work on it, since they already working some miracles, from my point of view, for us.
Can you tell as more about your experiment?
My impression was that Fortran is one of those languages where poor code is not that bad, though someone’s bound to prove me wrong. Part of it comes from its simplicity which also restricts it to a limited set of use cases in intensive number-crunching but not for general purpose programming
I share the same experience with you and I agree with you. Plus, Julia also enables me to tackle more hard problems/algorithms that I cannot even imagine I can even code with C++.
The point made in the video, that developers can produce quick and dirty code or highly optimized/high performance code, is applicable to every programming language I know, so, saying that Julia introduces this problem is incorrect. Even fortran, which is also sold as a fast language, allows you to write slow code, which I see all over the project I work on. Modern Fortran compilers are smart enough though to forgive developers some errors. On the other hand, even modern Fortran lacks many advanced features, which Julia has, and will never have these features because of its legacy.
The point is not that Julia introduces a new problem that other languages do not. (The reality is actually the opposite, you can do both these things better in the same language). The point is that this belief (and subsequent disappointment) from some newcomers is real. The anecdotes from the video and this thread show as much. I suspect that many that fall for this have, like myself, a science/engineering background rather than a programming background. If all you know is Matlab you don’t really have a grasp of what a language can or can’t do.
I’m not saying that anything needs to be done or changed, just that the presenter has a point. It is a real observable phenomenon.
This is exactly the point of the video, check the title:
I read this message literally and disagree with it. As for perception of Julia by newcomers, I agree, it may be frustrating at first, just because the whole Julia paradigm is so different from other languages, which some users got used to imo. May be documentation could do a better job explaining to newcomers what they can expect and what they can not. I am a newcomer myself and I can admit, that when I tried to solve my first real problem with Julia, I had some frustration as well, but not to the point that it turns me away.
May be confusing part for newcomers is an advertising that they can write a code as in Matlab and it will be as fast as Fortran code, which is not true, because Julia is so different from Matlab. Performant Julia code may be as concise as Matlab, or have somewhat similar syntax to Matlab, but it is not exactly as Matlab. Copying Matlab code to Julia 1-to-1 results in poor performance. This is something that newcomers should understand and adjust their expectations.
You are saying the same thing the video said. Like, exactly the same. There is an expectation mismatch. The video never poses the “1.5 language problem” as an actual problem to be solved. It is not a critique of Julia. It merely points out a real expectation mismatch that some newcomers experience that turn them off the language. The presenter argues we should address this. I’m not sure anything more needs to be done at the official level, but this realistic expectation is something I make sure to convey to people I’m trying to convert.
Final thought on this topic. Using the language from the presentation I do think Julia introduces a “1.5 language problem”(benign problem) precisely because it solves the two language problem. This is a good thing. The two language problem cannot be solved with a “single” language in the sense that rapid prototyping and performant code will always be different. But Julia comes as close as possible (no one else comes close). You can do both in the same language! But it is indeed two different efforts. As described by Chris R above.
The scenario Chris described is great but I think another good scenario is small research groups that do not have a “dev team”. Julia gives you a way to slowly make your code more performant while staying in the same language! This is a huge incentive for scientists/engineers to actually learn how to write performant code (journey I’m in) and learn more CS concepts (scientists/engineers need to accept we need more CS skills and while we are at it, math…).
That being said in an ideal world I would have both, my code look like the equations and run as fast as possible.
As newcomer to Julia, one of the first things I did was study the Performance page of the manual.
However, it did not prevent me from coming up with code like this, and wondering how this suddenly could take 40 seconds instead of two. After this, I learned that more compact, intuitive code is not necessarily fast:
(here, cellperpoint had 405_000 elements, activecells 3449 and indices is a BitVector that would select ~half the entries of cellperpoint):
@btime cellpoints = [cellperpoint[indices].== cell for cell in activecells]
3.032 s (31046 allocations: 4.92 GiB)
3449-element Vector{BitVector}:
while:
@btime cellpoints = [cellperpoint.== cell for cell in activecells]
472.810 ms (24148 allocations: 117.39 MiB)
It seems that Julia unnecessarily re-evaluated the indexing on every iteration. So my solution was to save cellperpoint[indices] to a variable first, then placing that in the comprehension, and get 236 ms, i.e. a 13-time speedup here.
Perhaps Matlab would have evaluated the indices once, before the iteration? Why couldn’t Julia do that automatically, given that neither variable changed within the comprehension?
is it possible that indices is a global?
Very sad statement. I hope that we can prosper in our niche regardless of that.
So around two years ago, I started a hobby project to use LLVM’s new JIT framework to produce small binaries for Julia (warning : it failed).
Crash course for Julia’s compiler in case that you don’t know how it works: Julia source code is firstly translated to IR (much like bytecode in Java). Type inferencer (implemented in Julia) adds type annotation to the IR. Then this typed IR is emitted to machine code by LLVM.
The workflow is like this:
And it turns out…the type inferencer itself was type unstable in several places, including some common mistakes like unhandled union splitting (Any is not splitted against complicated method signatures), closure and so on. So I can’t discard method tables for the type inferencer because dynamic dispatch needs to look up method tables.
I achieved this by a simple script because at that time Cthulhu.jl is not automatic enough at this task. :
for each IR in code_typed(function call):
if IR is a dynamic dispatch function call:
extract source code information from function's debug table
print human readable error
else if IR is static function call:
recur into function call if the signature combination is not seen before
end
end
It generates a list a possible error and source location. So in vscode I can quickly jump to the location and fix errors as many as possible, then start a new round.
I eventually gave up and decided to simply file an issue on github, because I didn’t want to patch Julia’s type inferencer – it complicated my workflow, and honestly speaking, I would rather rewrite the type inferencer in C++ instead of patching it in Julia, because it’s so inconvenient to boostrap the type inferencer and debug it.
Another experiement I did is to design a AST type inferencer for Julia (again, a simple 2000 lines type checker that can be implemented by any PL undergraduate). The lesson I learned from my previous experiment is that : IR type checker is essentially unusable because IR is not designed for human and the output is hard to interpret. So this time I set up some restricted syntax rules to systematically decide what should be considered as “type stable”.
I used it to scan some small packages, the result is surprising : many packages are not well typed and made small dump mistakes. Some people have hard time with parameterized type and forget to supply enough parameters. This is contrary to what people usually claim in the discourse. I fixed them manually by opening an issue on gihub. But then I got tired - it’s just not my duty to fix errors for other people and this way doesn’t scale. People should do this by themselves as they know more context than me.
I also tested it on my legacy computer graphics code, a tiny ray tracer. I caught two errors (one caused by type conversion and one by misspell) in the hot loop. IR checker simply can’t detect this because Julia silently does union splitting and the final result has no dispatch.
Anyway, none of these requires nontrivial PL techniques. Basically every of my PL friends know this as undergraduate, and one of them developed Julia’s Jetbrain plugin… That’s why I said a niche community is a huge problem. You need to spend more time and more effect to achieve the same thing…