Discussion on "Why I no longer recommend Julia" by Yuri Vishnevsky

I broadly agree, although I suspect that the dev base may 20x but the user base is more like 100x.

Well, but it is so much more fun to write Julia packages than Python packages, so this ratio is changing…

19 Likes

As it turns out julia is a Lisp after all, and so it suffers from it’s curse, that curse is also a blessing because I don’t know of another language where one can be so expressive and also write performant code at the same time.
On python there is a massive barrier between users and developers while in julia it seems that most users are an idea away of being developers.

23 Likes

14 posts were split to a new topic: Towards the creation of an interface testing package

I don’t think Julia is more dangerous than the early-stage JavaScript…
JavaScript has its advantage, then people who wants better software call for TypeScript.
Julia has its advantage, we might have something.

2 Likes

25 posts were split to a new topic: How do I know if a package is good?

A post was merged into an existing topic: How do I know if a package is good?

So is the picture that is emerging is that in Julia the SciML ecosystem will be the “equivalent” to Python’s Scipy? (i.e. the go to package for numerical computing, which includes most common problems)

If so, I see two fields which are still missing: eigenproblem, ffts. Are there plans to add those to the SciML ecosystem, or are those beyond scope?

(Perhaps this is off topic. Is soo, please fell free to move this to another thread)

1 Like

Yes, here’s the unified documentation (which is still in-progress, so don’t post it as something that’s complete, we’ll do a release announcement when it is):

https://docs.sciml.ai/dev/

Indeed you can see it’s meant to be the “go-to” for numerical computing. Indeed FFTs are missing, but that’s because it already existed and is good in Julia. I mention them in the docs:

https://docs.sciml.ai/dev/highlevels/interfaces/#AbstractFFTs.jl:-High-Level-Shared-Interface-for-Fast-Fourier-Transformation-Libraries

https://docs.sciml.ai/dev/highlevels/numerical_utilities/#FFTW.jl:-Fastest-Fourier-Transformation-in-the-West

But I wonder if there’s a better way to highlight them. Eigenvalue problems, yes that should get something similar to LinearSolve.jl but we just haven’t gotten around to it.

The other thing is interpolations.

22 Likes

I added FFTW just directly to the docs since it’s just the repo. That shold make it more searchable. DataInterpolations too (not Interpolations.jl because of course that is rough to use). Eigenvalue problems will take a bit more to do properly.

8 Likes

I don’t have much to add to this excellent discussion. I just want to say that it is great that Yuri and others opened all those issues listed in the blog post. Issues are actionable, and at least give developers the opportunity to address them, even if it occasionally takes a long time because of underlying difficulties or time constraints.

28 Likes

I think that having a “one-stop-shop” for most common tasks in numerical computing is really import in order to advance the adoption of Julia. Having a unified SciML ecosystem will greatly improve this! So thank you for your efforts @ChrisRackauckas !

I have been facing a problem when trying to convince students to use Julia.

First of all, I think Julia is a fantastic language! I love the simple syntax (much simpler than python’s which becomes a mess as soon you start using array). Types + multiple dispatch just immediately made sense to me (while python objects are hermetic, arcane nonsense to me).

But All of the students I interact with already know python+numpy+scipy (and many also know about numba).

How can I convince them to use Julia?

Well for-loops are horribly slow in python, while in Julia they are as fast as they can be. So that must be it! Except that there is numba, which solves that problem. (actually, it were the limitations of numba 6 years ago that made me move to Julia, but it seems that numba has greatly improved since then).

What about multithreading? Numba now also supports multithreading.

GPU programming? Again Numba + CuPy.

Now all of these things can be achieved in Julia. But at present, there is too much friction.

In my area of research the most common tasks are: 1) solving eigenproblems, 2) solving linear problems, 3) quadrature, 4) sometimes a fft or 6) an interpolation. If a student has to tackle those in Python, the student just has to write:

import numpy, scipy, numba, matplotlib

and that’s it.

In Julia? The student has to import one package for each task and I have to educate them on the differences between packages.

Should the student use ARPACK.jl, ArnoldiMethod.jl or KrylovKit.jl? What if we want inner eigenvalues? Well ARPACK.jl has shift-and-inverse but ARPACK.jl is also very fragile and prone to throw cryptic error messages. So ArnoldiMethod.jl or KrylovKit.jl it is. Except those have no shift-and-inverse implemented. So the student as to role its own. So what was a very simple task (calculating a few eigenvalues) becomes a lesson in numerical linear algebra. While I am going through this, the student has probably lost interest in this wonderful language and already solved the problem in python.

Plotting? Should the student use Plots.jl, PyPlot.jl, Makie.jl, VegaLite.jl, Gadfly.jl? Again in python, the student already knows there is matplotlib.

All of this is to say, that I really think that having a default, unified package for most numerical tasks will greatly improve this aspect and I believe it will make convincing students to adopt Julia much easier.

(sorry if this was a bit ranty)

26 Likes

I agree 100%. I wrote something similar in this post, and we discussed the package ecosystem a bit over in that thread.

To summarize: I think the solution is that we, as instructors using Julia, select a few packages for the purpose of teaching undergraduates and standardize on those for education.

In my opinion, there is an easy part and a hard part to this. The easy part is putting up a “Julia for Education” website somewhere listing the package selections (and other tips for students). (My hope is that we could have a unified source that replaces the “beginning of class Julia quickstart” handouts we all write individually, which needlessly duplicates effort.) I would be happy to do this if there is sufficient interest.

The hard part is making sure those packages are easy to use and bug free. So, for example, it seems someone would have to write shift-and-inverse for one of the two packages you mentioned.

However, I think completing the easy part would help focus attention on what needs to be done for the hard part.

If you would like to discuss further, please let me know and I can create a new thread.

6 Likes

I agree, I think some standardisation in the teaching tools is very important. But I also think that is import to suggest packages that are not just good enough for education, but can also be used for research (just like numpy+scipy). Otherwise, we end up with a kind of “two packages problem”.

That is why I am really hopeful regarding the SciML ecosystem as it will solve the hard problem.

That is what I made for personal usage with a “common interface” for different eigensolvers. Probably not good enough for public usage though, so I never bothered to put it up on github. But apparently SciML will include something like that

and with better code quality than I would be capable of.

This discussion is a tangent to the original topic, so a new thread is probably a good idea.

1 Like

Given this, I think the answer is probably “you don’t.” You either tell them to use Julia, in which case you should probably think really hard about your learning objectives when it comes to the code parts of your lessons, or you should stick to teaching the domain-specific stuff, and let them figure out the code stuff on their own.

When I say “think hard about the learning objectives,” I mean that you want to state explicitly (to yourself if nowhere else) what it is you want students to learn, then make you can assess that, and make sure you’re modeling it and giving plenty of practice and feedback.

For the most part, IMO, trying to convince students of things is a lost cause. You can share your excitement and hope some of it rubs off, but if you really want students to learn a thing, you need to explicitly teach it.

29 Likes

I don’t understand why you would want to do that.

Also, from the other person’s perspective, I would really hate it if users of other languages would spend any significant effort trying to convince me, personally, to use it. It would mean that I have to endure a lot of spiels about something I am not, at the moment, interested in.

If you are their professor, students are probably polite when you do this. Don’t let this mislead you, it may still be tiresome for them.

I am perfectly fine with using Julia. They are probably fine with their own favorite language. Live and let live.

21 Likes

Somethings that I should clarify regarding my comment:

  1. By students, I meant students I supervise (BSc final project, MSc, PhD), not students I teach classes.
  2. I don’t try to brainwash students into using Julia. I simply recommend students I supervise to use Julia: I let them know it exists (many don’t even know about it), say it is a nice language for scientific computing, that it is what I personally use and am more proficient with, and therefore, the language I will be more capable in providing support in case they face problems (but still provide all the help I can in whatever language they end up using). I also tell them to use whatever they fell more comfortable. If it works for them, it works for me!

Why? Simple: to increase the number of Julia users. In part it is a selfish reason: the more people around me use Julia, the easier it is to share code with colleagues. I believe education (either via classes or supervision) plays an essencial role in promoting adoption (wasn’t that the way python became so popular?).

Now, I don’t mean that no students pick up Julia. Some are interested in it and end up using it. But many, just continue using python+numpy+scipy. And that is fine! I just expected that convincing science (physics) students to adopt Julia would be much easier than it actually is. Isn’t Julia (I know it is general purpose) main target technical/scientific/numerical computing?

23 Likes

Students who work with me on their PhDs have no choice: I use Julia,
and the collaboration will be mediated by Julia (and the body of work already
implemented in Julia). If they wish to use other software for anything else,
of course they are free to do whatever they want.

9 Likes

Having someone to provide support is extremely valuable when learning something new, especially a human who’s nearby.

The Julia discussion forums are also very good in helping. You might not get good answers to all of your questions, but reasonable questions should work fine. This is very different to say Python or Kubernetes. (I’m using these examples as I have relatively recent experience with them.) With these other two I got the impression that the discussion forums were populated with beginners asking questions from each other or if there were experienced users, they were silent and making money with their knowledge.

This is probably related to how new and improving some tech is. For instance nushell has a very nice and active Discord community, and I’ve been wanting to use nushell more.

2 Likes

This definitely changes my perspective, and it sounds to me like you’re doing it right. Carry on :smile:

4 Likes