What don't you like about Julia for "serious work"?

Community
102

What don’t you like about Julia for “serious work”?

First of all, I would like to say that I like Julia and I like what the core team is trying to achieve. I like to play with Julia. To be honest for me it is more a toy than anything else. I am still struggling to find a real use for it or even use it.

If I need to prototype a model, glue things together do some stats, make some plots or create automated analysis and reports, then I can use R, Python or even Excel. Any of these 3 options will do a better job than Julia because they have better libraries, tooling and ecosystem in general. They also do not suffer from the Time to first plot, have better documentation and larger communities. That also means that you are more likely to find someone to help you solve your problems, understand code that was written by someone else or even find some to hire with that particular skill. I can list more things that are missing from Julia but that is not the point here.

Most people around me who tried to use Julia only see it as an immature fast version of R/Python. However for most people everything is not all about speed. Most people around me would say that they need to get things done quick, in a stable, reliable, secure and reproductible way. But often the Julia workflow is a pain, ecosystem too young, correctness issues, huge memory consumption and bugs like this one that drove me crazy:

That being said, which language is not a pain when you start learning it? None, they all have their good side and bad side… you just have to take the time to learn and appreciate them…

Also interacting with existing C++ libraries is key, because a lot has been built in that language. I know that Julia has CxxWrap, but it does not seem easy to use. Maybe I did not try hard enough or I am too stupid to manage to make it work. And there is no way a large company would rewrite its massive code base from C++ to a young language like Julia. That is never going to happen!

I really can see the benefit of Julia when someone has a need for prototyping something which must be fast and that also run for a long time. But that is really a niche for a language that aspires to be general. Other than that I am not sure what else can it be used for…

If someone or a firm wanted to build a software it would probably be better to build it in C++/Rust/Go/Java… all these languages can create small compact GUI, exe, dll …etc which can be used by other projects or by users with various level of IT skills. Yes Julia can create an (large) exe application (with 20 dlls) with PackageCompiler but again when it comes to creating an exe, app to be distributed, proprietary libraries, software, phone app…etc C++/Rust/Go/Java are better option than Julia.

Where does that leave Julia? In the middle between Python and C++, filling a small niche… And don’t get me wrong, you can do some serious quality work if you are in this niche…

If I had only 1 wish, I would ask for Julia to be as good as C++/Rust at creating exe/dll. That would be a big selling point over Python/R. The other “issues” like TTFP, slow compile time, lack of libraries… are minors issue from my point of view, and I would probably be able to live with them…

5 Likes
103

At my workplace three things.

  1. Researchers with no interest in computers find it too complicated compared to R and Python. They refuse to use git and write their own packages, and like their code in one file and don’t understand why all Julia code needs to be inside functions. They also love RStudio.
  2. The computer experts find it not as flexible as Rust (or other such languages).
  3. A lot of our code is run in pipelines on the command line, and may take 10 seconds to run, then a startup time of 60 seconds is annoying.
8 Likes
104

that nails it.

I would say the squeezed middle between {fortran, c, c++, rust} and {r, matlab, python}.

I’ve done projects in Julia, but when doing something small and new will almost always do it in python or r. And have started learning rust.

3 Likes
105

tracking type stability problems takes time and is not obvious. This is the #1 issue for HPC. For Julia wizards, it may be easy.

1 Like
106

Au contraire my friend: Julia is not a in a small niche in between, it spans the space from C and its ilk to Matlab and Python, and it is a closed interval (it overlaps those languages).

15 Likes
107

That is what I want Julia to be.

Alas, Julia is more complicated than the {r, matlab, python} of this world and not as powerful as {fortran, c, c++, rust}s.

Next time I win the lottery I will see if I can’t get the RStudio folks to add Julia to it, like they did with Python. Then Julia has a better chance there.

And I am hoping for standalone thin Julia binaries that one can easily copy and run on random systems.

Until then, the squeezed middle it is.

5 Likes
108

Next time I win the lottery I will see if I can’t get the RStudio folks to add Julia to it, like they did with Python. Then Julia has a better chance there.

They already did:

9 Likes
109

Reading datasets from cloud storage. These are often (not always) hive partitioned parquet files in a ‘data lake.’

There is often a metadata store, but this can be elided if you know the layout. It is effortful to build your own file tree to read into memory for some transform logic before writing elsewhere. Parquet is not well supported currently (kudos to the huge solo effort on Parquet2.jl), and its a ubiquitous data lake format since the schema is encoded with the conpressed data.

Its just a lot easier to run a spark cluster to handle that initial read for joins or filtering. So I’m generally managing two environments, a spark or MPP DB one (trino, redshift, etc) and wedging Julia work in there further downstream.

2 Likes
110

Incidentally, I was asked to evaluate and make a recommendation between Quarto and Jupyter books.

But, Quarto is not RStudio. And adding Julia (or any language) to Quarto is an order of magnitude easier than adding it to RStudio.

One can hope.

4 Likes
111

Maybe think about contributing to Parquet2.jl? Thats how this changes

2 Likes
112

Thanks for the hint! I see you just want the full RStudio (I am not exeprienced with) rather than just RMarkdown. IDE activities on the Julia side in the moment focus on Visual Studio code (which I don’t use as well as a diehard emacs user…) .

2 Likes
113

I don’t know if you speak from your own experience, but before you start boldly generalizing, you may wish to consider that your single data point may well be atypical.

I for one have created systems of decent complexity in all of these languages, except R. I found Matlab and Python fragile and disappointing overall.

And in my experience, Julia is much more powerful than C, C++, and Fortran when designing large systems. It is expressive, obvious, interactive, and high performance is reasonably easy to get. It aids in attaining reproducibility. Programming in C, C++, and Fortran is tedious, time consuming, and overall quite expensive in terms of man power.

20 Likes
114

Actually, I code in VScode and neovim.

But even R coder I know prefers RStudio. And its taught in the unis I know. And to get people started on coding, it is perfect. The gateway drug to R and now python. So, if it did Julia, would be fantastic.

115

This thread is quickly becoming unproductive but I’m surprised by this claim, surely Julia’s built-in ND Array and high-level numerical functions (as opposed to np.) and fast for loops (as opposed to remembering exactly what function you need every time) only makes it less complicated than these languages right?

From a user angle, they don’t need to interact with typing system much, so it’s just the surface syntax you’re talking about

13 Likes
116

I fully agree, and always council against taking too much from anecdotes. I am a statistician after all.

I like you have created decent sized systems in all of those, except fortran. I will resign and become a barista again before anther job in c or c++. Your experience is the same as mine. Julia is by far my favourite language to code in. I do not like python or r but I use both daily.

But I do have a decent sample of how colleagues/friends/partners/fellow students approach coding. So my take is perhaps more sociological than statistical. But here is what I have observed.

At the risk of overgeneralizing, the scientists I work with fall in two groups: those who reluctantly code, and those who love coding.

The former want maximum simplicity, and just getting the job done. The ones I have seen do not warm to Julia. (and they refuse to use git and the like, believe me, I tried)

The latter complain about ease of deployment and the like. they like single thin binaries. come from c/c++/fortran, moving to rust. Maybe its their arrogance, but I see them dismiss Julia, like they dismiss Python and R and Matlab.

So, in my sample, it it not a question of the language, it is all the other things.

and my sample tells me Julia is being squeezed in the middle by those two.

I wish this was not my sample and wish I had a way to sway those two groups

6 Likes
117

Yes, It is not the language as such (well the huge library ecosystem in R/Python) helps.

It is how one uses it.

and RStudio is the easiest way anybody can start coding. I think RStudio is the single best advantage R has.

I have delivered statistical programming/data science training with R/matlab/python/Julia

R+RStudio is by far easiest on the students,

2 Likes
118

I do not see why a non-programmer person would need to use GIT at all. And, at the same time, I think these difficulties are mostly due to differences in habit and experience with packages then real advantages of Python/R: Julia syntax is much clearer, and the cost of having to implement things in functions (which is not even mandatory) is smaller than having to adapt one’s code and thought to every lower-level routine available. What is true is that newcomers tend to iterate by re-running scripts all the time, and that goes against Julia’s best workflow. Particularly if one is, for instance, just plotting stuff, not doing that is anti-natural, and that goes against Julia’s first impressions.

Binaries yes, they are great, and the static analysis tools for static languages are a must. But there is nothing easier than deploying in Julia, as it just depends on structuring the code as a package and uploading it to GIT.

4 Likes
119

I have to disagree. I can make a compiled c/c++/rust/fortran binary, and just copy it onto any system I want. then its one command to run. Not the same with first installing Julia (or R or python), all the libraries, and then do a git pull. And then running by “julia code.jl” instead of clicking on it or linking with whatever they are using. I have to do this for work on our systems. and much prefer the single binary deployment.

5 Likes
120

Sadly not only newcomers. I have a superstar old scientist on my floor, and this is how he has worked for longer than I have lived. And he does stellar work. (not stellar code, but his code makes the work)

3 Likes
121

Ok, on that sense, yes. I meant in the sense of deploying your code to all major platforms, in a way that anybody anywhere can run. For me having to release binaries for other platforms than the ones I use (Linux) in Fortran has always been a headache. I much prefer to tell users to just install julia, add MyPackage and start using the code.

I meant newcomers to Julia. And I certainly agree with all the rest, my dad is one of those :slight_smile: (superstar is probably exaggerated, but he’s one the most accomplished mathematicians of Latin America, thus I can give him the credit, as a son).

11 Likes