Julia's applicable context is getting narrower over time?

We recently added Julia to our codebase, where we’d previously used Python only for 3 years. This was motivated by optimizing a major process that was a key part of our product and cost $15k/month to run. As you can imagine, we had already heavily optimized it in Python. Everything was using numpy and calling C libraries beneath the hood, but besides that we employed many performance tricks that resulted in very ugly and unpythonic code. We had several very talented engineers look at this, including one who specializes in optimizing numeric Python code.

In a few days of work, using Julia for the first time, I was able to produce a prototype that was 14 times faster than the heavily optimized Python code. There are a lot of reasons for that, but a few of the main ones were Julia’s light-weight threading model and the fact that profiling in Julia is tremendously easier, since it’s “Julia all the way down.” A few weeks later that code is in production, and has seen another ~2x speedup for a total of ~30x. As far as I can tell, the improvements I made wouldn’t be possible to replicate in Python right now. The fact that we spent 200+ expert hours on the Python version vs. 10 hours of a beginner’s time on the Julia also tells you a lot about the developer experience.

So I don’t think Python is narrowing Julia’s niche – if anything, Julia’s niche is currently expanding as more and more libraries are developed.

106 Likes