Julia: a post-mortem

One (niche) doesn’t exclude the other (general purposeness). Julia was started and is still advertised as a language for technical computing (stats, data science, simulation, etc.) that is (a lot) faster and easier to use than existing languages.

3 Likes

This is the kind of thing that you can pass a life programming and not knowing that it exists if using other languages (saying for myself). And that is not in favour of the other languages, codes are slow in any language if you do not take care of that.

I learned while programming in Julia that even a Fortran program can allocate an intermediate variable in a loop if you do not set the proper compilation flag, for example. I was getting faster Julia codes than Fortran and it was Chris Elrod who showed me how to get to the same speed in my Fortran code. Also, now when I see a numba, pypy or other accelerated python code, I can understand where they will not get to full speed because of improper type inference that may arise. So the concepts that you need to learn are there in all languages if one wants to achieve the ultimate performance.

After 20 years writing Fortran in collaboration with many other researchers, those things never came up. We were just doing reasonable, but suboptimal, code without knowing it. Of course if one comes from a strong computer-science background community the experience will be different.

Julia allows one to learn those concepts in the context of a high-level language which is practical and pleasant to code with. That is why I think learning it is very nice, even if one cannot use it at the end.

29 Likes

Mhm, this is an especially interesting point to me as it can cut both ways. Yes, writing high performance julia code often requires quite a bit of knowledge and understanding, probably comparable to languages like C. But on the other hand I find that the language design draws me in and encourages me to get down into nitty gritty programming details and learn things I’d have likely never learned back when I was a Python and Mathematica user.

I think the big difference is that in say Python, dropping down to C introduces a big potential barrier that keeps me away from learning about internals, whereas in Julia I’m constantly encouraged to just dip my toes into something that’s outside my comfort zone and learn something new about low level details in bite sized chunks. Because I can do this without purposefully learning a new language, I almost accidentally became rather proficient with high performance julia code and compiler passes.

I never intended to learn most of the things I’ve learned about julia over the last few years, it kinda just sucked me in and turned me from a very naive scripter to someone rather passionate and interested in programming generally.

39 Likes

When discussing relative language popularity, please, please, please don’t use TIOBE as a source of anything meaningful — it’s methodology is so deeply flawed it means hardly anything. Instead, use rankings with reasonable methodologies like IEEE and PYPL. According to PYPL, which was the most pessimistic about Julia’s ranking at the year’s start (TIOBE had Julia at #23 at the time), Go is 3.8x more popular than Julia. But Python is 23x more popular than Go. Yet nobody is wringing their hands over Go’s failure or lack of growth. Considering Go’s three year head start and massive corporate backing from Google, the actual question it seems like people ought to be asking is “how has Julia, without any large corporate sponsorship, been so wildly successful in such a short time?”

50 Likes

Julia came into being as a solution to the “two language problem” in scientific computing, which is that the workflow of many (most?) scientists involved working in an interactive scripting language and getting over the performance problems by linking to precompiled libraries written in a statically compiled language.

Most languages are fairly general purpose. There have been webservers written in MATLAB.

I don’t think the developers have ever claimed that Julia would replace all languages. Other parts of that post-mortem would have to be expanded upon (e.g., the claim that 1 based indexing leads to suboptimal code, issues with multiple dispatch, …) to make that critique more solid.

3 Likes

I think this post explains it better than I could.

There have been webservers written in PostScript… :wink:

2 Likes

Trust me, I have been using Julia on and off for the past nine years and I have been there and overanalyzed these things to death. During this time I mostly used Julia for personal side projects and only for a very short time (~18 months) professionally as part of my day job.

I have come to the conclusion that whether or not a programming language is right for me is highly subjective and pro-contra articles and statistics wont help me. Reading more will only give me analysis paralysis.

You only get a feel for a language and ecosystem when you use it to build a real project. Doing exercises, reading books and doing courses doesn’t cut it in my opinion. Because if you are following along the synthesis step is missing and that is where the magic happens.

Give yourself a fixed amount of time (4 weeks maybe) and go for it. Try to go on until you have grokked the central paradigm (Multiple Dispatch in Julia’s case) because that is the point where the fun begins. Then ask yourself, did you enjoy the process? Did the language help you solve your problems?

If you did not enjoy yourself, there is no point in continuing. I guarantee you that the time will not be wasted even if you drop out. You will learn about new concepts that can be applied elsewhere. If you did enjoy yourself but could not solve some problems, ask yourself whether the language could get there in time or wait for a different problem :wink:

As for the statistics and the perceived viability of a programming language, only time will tell and open-source languages never just disappear. If statistics mattered (no. of devs, lines of code deployed, available libraries), we should probably all be using Java.

Bottom line

  1. Work on a non-trivial project with language X
  2. Check whether you had fun
  3. Continue or move on
  4. Profit!

Even though I wish Julia had Rust’s static binaries, Elixir’s runtime, Elm’s simplicity, or Python’s gigantic community, I keep coming back to Julia because it is the most fun and fits my brain best.

34 Likes

The line from this article that resonates with me:

“the yaks have largely been shorn and shaven, and you won’t have to bother with them”

I really like Julia and use it as often as possible. As a fancy calculator. The fanciest best calculator I could hope for. I don’t use it as a programming language for creating software though. Much software today is built out of pieces from more than one source to run on multiple platforms. Julia can’t solve the two language problem unless it puts some effort into integrating itself seamlessly into a multiple language software stack.

3 Likes

And that is a contradiction in itself :slight_smile:

(of course more interoperability is always welcome - but certainly Julia cannot nor aim replacing everything that is written every other language, but the “two” language problem as understood in the narrow context of begin able to prototype and optimize new problems in the same language, it can)

2 Likes

My job is to build new software. I do solve problems in Julia, my trusty calculator. Then I need to write software in C++ or python because they are seamlessly integrated into the software/OS environment of the end-users.

1 Like

Julia already does a great job incorporating other languages (PyCall, RCall, CxxWrap, native ccall…), which was critical as a way of overcoming the deficits of a young-language ecosystem, but it’s not yet easy to go the other direction. We’re already reaching the point where Julia’s strengths have allowed the development of world-class software for a few subdomains, and there’s demand for that software outside our language silo that we’re currently unable to meet. That’ll probably have to wait until the binary-compilation story improves to the point where it’s no more than twice as hard as gcc -c. Small static binaries are near the top of the core devs’ priority list, though, so it hopefully won’t be long.

14 Likes

Julia can’t solve the two language problem unless it puts some effort into integrating itself seamlessly into a multiple language software stack.

Interesting. I think one point that’s often undersold about Julia is how well it integrates with other languages and environments: RCall.jl, PyCall.jl, MATLAB.jl, etc… work pretty well. I personally have found that makes it often pretty easy to integrate with larger software stacks (at least when compared to other cross-language situations). It’s the other direction (e.g. making a portable, standalone julia binary or library with a small footprint), which I think you are mostly referring to, that can pose more challenges (but even that’s getting better).

That said, I think if you are building a tool along a well trodden path, where there are a bunch of libraries in language A to accomplish your task, the best way to use those is usually to use language A, not call it from your favorite language B. B might be a more beautiful and satisfying language to write in, but it will still take more time if there are a bunch of really effective tools already extant in A. I think one of @leadromartinez98’s points was that when you are journeying along less well trodden paths that the power and beauty of a language becomes more of an advantage.

Edit: whoops, I @stillyslalom took the words out of my mouth before I finished typing. Sorry for the redundancy.

4 Likes

I am yet to see a non-trivial piece of C++ software to “seamlessly integrate” anywhere. In fact, in my experience it is anything but.

9 Likes

Genuine interest: in what ways does Julia lack integration with other software/OS? And in what way is that situation better with other languages?

1 Like

I agree – I’ve been rewriting performance critical parts of our Python codebase in Julia lately, and it’s been pretty seamless to either call the Julia functions from Python using PyJulia, or vice versa using PyCall. I’d say it was roughly a one-time investment of two hours to learn how to get these working and get everything set up properly.

1 Like

The biggest here is static binaries. You really need those if you want to distribute software (especially because most OS update less frequently than Julia, so shipping Julia in the OS is a bad idea).

4 Likes

@orbots mentioned Python in the context of my question, which has no (portable) way of distributing static binaries. Plus it’s not a very common way of distributing Python code anyway. So I’d be surprised if this is the biggest hurdle for distribution, especially in an open-source context?

1 Like

I think the difference here is that very few people make Python libraries to be called from other programming languages. This would not be very efficient due to Python’s performance limitations. For Julia, however, this would be a viable use case.

1 Like

Interestingly, this index introduces the C programming language suitable for mobile development. Or it introduces Arduino as a programming language (which is fine) but on the other hand it only introduces JavaScript suitable for web development and does not even look at NodeJS separately as if that part of the development world does not exist at all. This index does not seem to be very valid either.