Julia: a post-mortem

I am very familiar with Python. Please give me an example to understand what you are talking about.

It’s the niche. Julia competes with similarly easy to use langages. Go and Kotlin have niches in the web and JVM, respectively, and generally compete with harder to use languages.

1 Like

Threads like this come along pretty frequently and I pass them by because they aren’t productive, but today is the day that I give my unsolicited opinion. As background, I am proficient in R/MATLAB/Python/Lua, conversational in C derivates, and have some low level application hacking experience but no didactic comp sci training. I have been using Julia for more than a year now in industry.

I don’t find the first argument in the linked post compelling. The ecosystem of Julia isn’t as big as Python, but it’s big enough for my needs and people love rewriting packages so I am sure it will keep growing. If anything, the lack of R-like package proliferation and general openness of the community is a selling point for me.

To engage with the second point (cui bono?), what drew me to Julia was the promise that if we used Julia, my team and I could avoid rewriting scripts in C for performance reasons. I was attracted to a workflow where I could try stuff out using inefficient code (the allure of a scripting language) and I would just need to spend an afternoon tightening the screws when I want to optimize. Apart from personal workflow benefits, this makes full stack / near-realtime development accessible to scripters, which in turn hits a sweet spot in the employee spectrum where anyone with some scripting experience can contribute to a project potentially right from day 1 (1-indexed forum). When hiring, specialty + scripting experience is a much wider net than specialty + serious developer experience. It seemed like setting up a Julia-based shop would reduce or eliminate the need for explicit developers on greenfield research projects.

If I was going to argue the bear case for Julia with the 2019 version of me, it would be that writing Julia efficiently enough to avoid having to rewrite your script in C for performance reasons requires a similar level of technical knowledge as coding it in C. You ultimately need someone with a fairly strong comp sci background that understands when allocations happen in the network of Julia packages being used by the script and can interpret lowered code. So you don’t really get to skip the developer skill set and in addition your developer has to be fairly knowledgeable about the Julia package ecosystem used by the scripts. To me, this is a longer term and more serious problem for the Julia sales pitch than “the ecosystem is immature and we’re not sure it will attain escape velocity.”

I am not moving away from Julia in the foreseeable future and if I am honest with myself I already have all the tools I need to solve my own problems. If I wanted to do more than complain, I would get involved with the profiling projects and try to make them (perhaps just the docs) more accessible to people with weak-to-moderate scripting backgrounds.

36 Likes

The main languages that julia seems to substitute for me are Matlab and Fortran. It doesn’t have the licensing costs of Matlab and it is as powerful as fortran without the high entry barrier associated with doing high performance computing in fortran.

7 Likes

What niche are you talking about? Where this niche is defined? As I understand it, Julia is a general-purpose programming language.

Go and Kotlin are both a lot older than Julia (8 and 5 years since 1.0, compared to 2 1/2) and backed by Google.

3 Likes

Haskell is a general purpose language and people dont call it dead even if it its restricted to a couple of niches.

One (niche) doesn’t exclude the other (general purposeness). Julia was started and is still advertised as a language for technical computing (stats, data science, simulation, etc.) that is (a lot) faster and easier to use than existing languages.

3 Likes

This is the kind of thing that you can pass a life programming and not knowing that it exists if using other languages (saying for myself). And that is not in favour of the other languages, codes are slow in any language if you do not take care of that.

I learned while programming in Julia that even a Fortran program can allocate an intermediate variable in a loop if you do not set the proper compilation flag, for example. I was getting faster Julia codes than Fortran and it was Chris Elrod who showed me how to get to the same speed in my Fortran code. Also, now when I see a numba, pypy or other accelerated python code, I can understand where they will not get to full speed because of improper type inference that may arise. So the concepts that you need to learn are there in all languages if one wants to achieve the ultimate performance.

After 20 years writing Fortran in collaboration with many other researchers, those things never came up. We were just doing reasonable, but suboptimal, code without knowing it. Of course if one comes from a strong computer-science background community the experience will be different.

Julia allows one to learn those concepts in the context of a high-level language which is practical and pleasant to code with. That is why I think learning it is very nice, even if one cannot use it at the end.

32 Likes

Mhm, this is an especially interesting point to me as it can cut both ways. Yes, writing high performance julia code often requires quite a bit of knowledge and understanding, probably comparable to languages like C. But on the other hand I find that the language design draws me in and encourages me to get down into nitty gritty programming details and learn things I’d have likely never learned back when I was a Python and Mathematica user.

I think the big difference is that in say Python, dropping down to C introduces a big potential barrier that keeps me away from learning about internals, whereas in Julia I’m constantly encouraged to just dip my toes into something that’s outside my comfort zone and learn something new about low level details in bite sized chunks. Because I can do this without purposefully learning a new language, I almost accidentally became rather proficient with high performance julia code and compiler passes.

I never intended to learn most of the things I’ve learned about julia over the last few years, it kinda just sucked me in and turned me from a very naive scripter to someone rather passionate and interested in programming generally.

41 Likes

When discussing relative language popularity, please, please, please don’t use TIOBE as a source of anything meaningful — it’s methodology is so deeply flawed it means hardly anything. Instead, use rankings with reasonable methodologies like IEEE and PYPL. According to PYPL, which was the most pessimistic about Julia’s ranking at the year’s start (TIOBE had Julia at #23 at the time), Go is 3.8x more popular than Julia. But Python is 23x more popular than Go. Yet nobody is wringing their hands over Go’s failure or lack of growth. Considering Go’s three year head start and massive corporate backing from Google, the actual question it seems like people ought to be asking is “how has Julia, without any large corporate sponsorship, been so wildly successful in such a short time?”

53 Likes

Julia came into being as a solution to the “two language problem” in scientific computing, which is that the workflow of many (most?) scientists involved working in an interactive scripting language and getting over the performance problems by linking to precompiled libraries written in a statically compiled language.

Most languages are fairly general purpose. There have been webservers written in MATLAB.

I don’t think the developers have ever claimed that Julia would replace all languages. Other parts of that post-mortem would have to be expanded upon (e.g., the claim that 1 based indexing leads to suboptimal code, issues with multiple dispatch, 
) to make that critique more solid.

3 Likes

I think this post explains it better than I could.

There have been webservers written in PostScript
 :wink:

2 Likes

Trust me, I have been using Julia on and off for the past nine years and I have been there and overanalyzed these things to death. During this time I mostly used Julia for personal side projects and only for a very short time (~18 months) professionally as part of my day job.

I have come to the conclusion that whether or not a programming language is right for me is highly subjective and pro-contra articles and statistics wont help me. Reading more will only give me analysis paralysis.

You only get a feel for a language and ecosystem when you use it to build a real project. Doing exercises, reading books and doing courses doesn’t cut it in my opinion. Because if you are following along the synthesis step is missing and that is where the magic happens.

Give yourself a fixed amount of time (4 weeks maybe) and go for it. Try to go on until you have grokked the central paradigm (Multiple Dispatch in Julia’s case) because that is the point where the fun begins. Then ask yourself, did you enjoy the process? Did the language help you solve your problems?

If you did not enjoy yourself, there is no point in continuing. I guarantee you that the time will not be wasted even if you drop out. You will learn about new concepts that can be applied elsewhere. If you did enjoy yourself but could not solve some problems, ask yourself whether the language could get there in time or wait for a different problem :wink:

As for the statistics and the perceived viability of a programming language, only time will tell and open-source languages never just disappear. If statistics mattered (no. of devs, lines of code deployed, available libraries), we should probably all be using Java.

Bottom line

  1. Work on a non-trivial project with language X
  2. Check whether you had fun
  3. Continue or move on
  4. 

  5. Profit!

Even though I wish Julia had Rust’s static binaries, Elixir’s runtime, Elm’s simplicity, or Python’s gigantic community, I keep coming back to Julia because it is the most fun and fits my brain best.

37 Likes

The line from this article that resonates with me:

“the yaks have largely been shorn and shaven, and you won’t have to bother with them”

I really like Julia and use it as often as possible. As a fancy calculator. The fanciest best calculator I could hope for. I don’t use it as a programming language for creating software though. Much software today is built out of pieces from more than one source to run on multiple platforms. Julia can’t solve the two language problem unless it puts some effort into integrating itself seamlessly into a multiple language software stack.

3 Likes

And that is a contradiction in itself :slight_smile:

(of course more interoperability is always welcome - but certainly Julia cannot nor aim replacing everything that is written every other language, but the “two” language problem as understood in the narrow context of begin able to prototype and optimize new problems in the same language, it can)

2 Likes

My job is to build new software. I do solve problems in Julia, my trusty calculator. Then I need to write software in C++ or python because they are seamlessly integrated into the software/OS environment of the end-users.

1 Like

Julia already does a great job incorporating other languages (PyCall, RCall, CxxWrap, native ccall
), which was critical as a way of overcoming the deficits of a young-language ecosystem, but it’s not yet easy to go the other direction. We’re already reaching the point where Julia’s strengths have allowed the development of world-class software for a few subdomains, and there’s demand for that software outside our language silo that we’re currently unable to meet. That’ll probably have to wait until the binary-compilation story improves to the point where it’s no more than twice as hard as gcc -c. Small static binaries are near the top of the core devs’ priority list, though, so it hopefully won’t be long.

14 Likes

Julia can’t solve the two language problem unless it puts some effort into integrating itself seamlessly into a multiple language software stack.

Interesting. I think one point that’s often undersold about Julia is how well it integrates with other languages and environments: RCall.jl, PyCall.jl, MATLAB.jl, etc
 work pretty well. I personally have found that makes it often pretty easy to integrate with larger software stacks (at least when compared to other cross-language situations). It’s the other direction (e.g. making a portable, standalone julia binary or library with a small footprint), which I think you are mostly referring to, that can pose more challenges (but even that’s getting better).

That said, I think if you are building a tool along a well trodden path, where there are a bunch of libraries in language A to accomplish your task, the best way to use those is usually to use language A, not call it from your favorite language B. B might be a more beautiful and satisfying language to write in, but it will still take more time if there are a bunch of really effective tools already extant in A. I think one of @leadromartinez98’s points was that when you are journeying along less well trodden paths that the power and beauty of a language becomes more of an advantage.

Edit: whoops, I @stillyslalom took the words out of my mouth before I finished typing. Sorry for the redundancy.

4 Likes