arXiv: "The State of Julia for Scientific Machine Learning" by Berman & Ginesin

Specific Domains Machine Learning
, ,
22

@world-peace, I can no longer quote your post, but to answer you, there’s e.g. this use case:

To a Python developer, SciML is SciPy, but with the high-performance GPU, capabilities of PyTorch, and neural network capabilities, all baked right in. With SciML, there is no “separate world” of machine learning sublanguages: there is just one cohesive package ecosystem.

Why SciML? High-Level Workflow Reasons

  • Performance - The key reason people are moving from SciPy to Julia’s SciML in droves is performance. Even simple ODE solvers are much faster!, demonstrating orders of magnitude performance improvements […]
  • Package Management and Versioning - Julia’s package manager takes care of dependency management, testing, and continuous delivery in order to make the installation and maintenance process smoother. For package users, this means it’s easier to get packages with complex functionality in your hands.
  • Composable Library Components - In Python environments, every package feels like a silo. Functions made for one file exchange library cannot easily compose with another. SciML’s generic coding with JIT compilation these connections create new optimized code on the fly and allow for a more expansive feature set than can ever be documented. […]
  • Easier High-Performance and Parallel Computing - With Julia’s ecosystem, CUDA will automatically install of the required binaries and […] Basically, libraries give you a lot of parallelism for free, and doing the rest is a piece of cake.
  • Mix Scientific Computing with Machine Learning - Want to automate the discovery of missing physical laws using neural networks embedded in differentiable simulations? Julia’s SciML is the ecosystem with the tooling to integrate machine learning into the traditional high-performance scientific computing domains, from multiphysics simulations to partial differential equations.
    […]
  • Use PyCall.jl to more incrementally move code to Julia.

It seems the docs are a bit outdated, I would not recommend PythonCall.jl rather than PyCall.jl (though both work even together is you’re careful).

You can use SciML alone from Julia (e.g. abandon Python), or you can clal it from Python (or R) with:

That package used to be implemented differently until I suggest julicall/PythonCall.jl. A lot of stuff online is outdated that way (not rather mentioning it):

In the space of symbolic regression, Julia has a clear advantage in terms of support. The most widely adopted library for symbolic regression is SymbolicRegressions.jl [17]. The Python alternative PySR is simply a wrapper for this library, which is in turn a dependency for PyTorch [62].

1 Like
23

9 posts were split to a new topic: Asides on “The State of Julia for Scientific Machine Learning”

32

To maybe get this thread back on track:

Here’s an example for a Python package using Julia as a numerical backend: QuanEstimation.

It’s perhaps not the most common use case, but I wouldn’t call it “niche”.

There is always value in being able to call a library in one language from any other language. Not the least of which, simply that that library might implement some very non-trivial algorithm, which you may not want to re-implement in a new language. Presumably, the semantics of the original language were particularly well-suited for the problem at hand, so it may not even be desirable to rewrite it in some other language with less-suitable semantics. Even if time and effort were not a factor.

I have no trouble imagining someone wanting to call the SciML ecosystem from inside C++.

7 Likes
33

One of the most popular Julia packages (at least early on) is Pyplot.jl, which allows you to interface with matplotlib, through Python. This is a typical example, calling libraries in a different language to get access to specific functionality.

3 Likes
34

I have called julia from C a couple of times. The reason was that it was much simpler to express the transformations and data generation I needed in julia than in C. Unfortunately I can’t go into any details about that project.

1 Like
35

Apologies to all those who were trying to continue the civil discussion here. It appears some people had quite the absurd overreaction to the most harmless comment.

I don’t feel like commenting any further because I don’t want to get banned, but I will just respond to the matplotlib comment because I think I have something beneficial to add here.

Just to say when Julia becomes more established, it might be the case that calling matplotlib directly is no longer required. I’ve always considered this to be a temporary thing. I’m sure someday soon someone will come up with something equally as good, written entirely in Julia.

36

Apologies to all those who were trying to continue the civil discussion here. It appears some people had quite the absurd overreaction to the most harmless comment.

I don’t feel like commenting any further because I don’t want to get banned, but I will just respond to the matplotlib comment because I think I have something beneficial to add here.

The issue was put into an unlisted thread for a reason, there’s no need to continue harping on and blaming others about it.

Just to say when Julia becomes more established, it might be the case that calling matplotlib directly is no longer required. I’ve always considered this to be a temporary thing. I’m sure someday soon someone will come up with something equally as good, written entirely in Julia.

Makie has already come pretty far for this especially. I wouldn’t say matplotlib is required at all for using Julia for most things (any?) at this point really.

4 Likes
37

Absolutely, I would not use Pyplot.jl over Makie at all! I just mentioned it as a well-established and known example of a use case for calling into other dynamic languages. And I don’t think the usefulness of doing that will disappear, even though it might be less common.

3 Likes
38

Hello! Jake and Eddie (the authors of this paper) here.

First of all, we’re at NeurIPS 2024! We’re giving an oral presentation on this work at the Machine Learning for the Physical Sciences, and we’ll also be participating in the workshop’s poster session. Come talk to us!

Second, we’ve been closely monitoring this thread and selectively emailing some of you to gauge further opinion, etc. We genuinely appreciate all the engagement and feedback — both the positive and the negative — on our work. We’ve incorporated a fair share of the feedback into the camera-ready version of the paper, which is available on the workshop website. We’d also be happy to update the arxiv version if there is interest.

On a personal note, we are both very happy to see such passionate conversation about Julia. We both love the language and want to see it improve.

Cheers,
Jake and Eddie :rocket:

42 Likes
39

Welcome to the community! Thank you for your work, and glad to see you here

7 Likes
40

Fantastic! Is https://ml4physicalsciences.github.io/2024/files/NeurIPS_ML4PS_2024_42.pdf the latest version?

If so, I do spot the mildly inaccurate claim I previously flagged that “Julia has virtually no support for more advanced testing methods such as property-based testing” is still present despite the existence and use of Supposition.jl.

It also would have been nice to see mention of Enzyme.jl under AD, since that’s seeing a lot more use these days, but I appreciate that there isn’t room for everything that could be said in the paper :slight_smile:

7 Likes
41

hey @JakeGinesin, i saw you cited Reactant.jl so thanks! you guys are the first ones to cite us. we’re talking about doing a first publication on Reactant in a couple of months, but i guess that your publication will go first.

Julia’s lack of software engineering-centric features, lackluster debugging infrastructure, subpar industry adoption, its “rivalry” with Jax, and insufficient interopt all bottleneck further adoption. However, none of these limitations in isolation seem like dealbreakers. On paper, Julia should compete with Python. So again we ask, why doesn’t it?

i think you might want to recheck Reactant.jl for the “‘rivalry’ with Jax” because it literally is like a “Jax for Julia on steroids”. we’re still not there but we’re slowly approaching and allows for interopt with Jax (checkout Calling Python function JIT compiled with JAX from Julia without overhead - #9 by sefffal for calling Jax traced functions with Reactant, and Exporting Lux Models to Jax (via EnzymeJAX & Reactant) | Lux.jl Docs for the other way around).

@avikpal is working hard on making (F)Lux + Reactant a real competitor to Python’s alternatives like Jax and PyTorch

We believe the current state of Julia lacks vision. Julia needs a new constitution: a set of concrete goals for improvement, adoption, and outreach. While Python has a clear list of future goals [2], and individual ecosystems within Julia such as SciML have roadmaps [74], the Julia language itself has nothing but surface-level GitHub issues. It is only once we map out and solve these language-level issues that we can then ask if Julia is capable of succeeding Python as the de-facto language for scientific machine learning.

We ask the Julia and scientific machine learning communities: what is the future for Julia?

this has been kind of discussed other times and the conclusion is roughly that it won’t work. i’m relatively new to the community, but i had a similar impression to you when i started and want to clarify what i’ve learned these last couple of years. IMHO writing a long-term roadmap for the language has several problems:

  1. it can will create false expectations of what’s next to come. just check out what has happened with GATs on rust. some features seem “easy” in the beginning but then need years of work.
  2. we don’t have the answers for many things, so we shouldn’t fix to a “solution”. instead, Julia mindset is more like “let users experiment with different proposals until we arrive to some kind of consensus or we find that it requires language support to work properly”

i feel like point 2 should be better explained. there are topics that are very very complex because they have a huge design space and have seen a lot of heated debate over a lot of years, so we haven’t reach consensus. this kind of topics don’t see much progress because there are a lot of tradeoffs and implementation details to be considered to even take a decision. one of this topics i would say is interface dispatch: should we go on multiple inheritance? should we formalize traits? we don’t know what we need in Julia to make this work properly.

but when after some experimentation some common conclusions start to emerge, then we can start thinking about what new features Julia needs and we can construct some kind of roadmap. for example:

  • static dispatch, started from the desire to call Julia generated code outside of Julia without the cost of JIT compilation. a lot of experimentation was performed first in PackageCompiler.jl
  • compiler plugins (and custom abstract interpreters), started from the desire to customize Julia’s compiler to write your own compiler transformations, optimizations and codegen backends. it is critical for some communities like GPU and autodiff, but progress has been “steppy”. this one has seen a lot of experimentation (and it’s still considered very experimental) in different packages but check Cassette.jl, GPUCompiler.jl, Mixtape.jl, Enzyme.jl, Zygote.jl…

summarizing, making a long-term roadmap for Julia is hard but short- or even mid-term roadmap is more or less implicitly known by just being around and checking the github issues and prs.
the closest things we have to a roadmap, where you see the progress on the topics i’ve mentioned and more, are the annual “State of Julia” talk at JuliaCon and the monthly “This month in Julia world”

27 Likes