Blog post: Rust vs Julia in scientific computing

Why I think that Julia doesn’t solve the two-language problem and when you should use Rust instead.

The blog post is the base for my tiny talk at Scientific Computing in Rust 2023 which is now public:

Think of the recording as a trailer. The blog post has many more details and aspects that can not fit into 7 minutes :hourglass:

I hope for discussions across both communities which is why I post here :smiling_face_with_three_hearts:

Sorry for creating a new thread. I know about other threads like this, but I did spend days on the blog post and would like to have a separate thread for its discussion :slight_smile:

22 Likes

That kind of topic: :popcorn:

My answer in one word: cherry picking.

6 Likes

I am pretty open minded on the resulting conclusions, but if you use the word “scientific computing” in your post I think you want to show what the code looks like for actual scientific computing algorithms, not artificial setups like having lists of Particles. etc. Solve an ODE, for example, completely using rust along with all of the parameter choices/etc. and using core packages for solving linear systems of equations. Show parallelization of some practical algorithm and show the differences in code.

This isn’t saying you are necessarily wrong on any of your points, but I don’t think you have quite captured the tradeoffs in this problem domain. Many people like Julia because it looks clean and like math (i.e., a better matlab rather than a better C++) and you can’t see that until you solve a real problem.

On the content itself you say Julia is planning for a 2.0. I haven’t seen any evidence of that at all. If things are pushed to version = 2.0 in Julia github it probably just means the idea is parked for the foreseeable future.

36 Likes

To get back to the initial question: Does Julia solve the two-language problem?

For me, the answer is: No

Although Julia has a just-in-time compiler that can make it very efficient, it misses the advantages of a real compiler for a statically typed language.

I guess it depends on how you define the the two-language problem, but what you have done is to show that Julia cannot compete with Rust in terms of static analysis, what makes total sense, Julia is a dynamic language, Rust is a static one. However, you did also not show that Rust does a better job at solving the two-language problem. Rust may be better on the “language for large system” side of the gradient, but it is a poor choice for the opposite side (prototyping). No language will ever solve the two-language problem, they will just solve it better than another language. And what is better depends on how you weight each side of the gradient. If you put all weight in the “large scale system” side of the gradient, then Rust seems a better choice, but the question should not be if Julia solves the two-language problem (as an absolute) nor the answer “no, because Rust does some things better at this side of the gradient”. The question should be if Julia solves it better than Rust (or some other language), what is not shown there (only one side of the gradient is examined). Languages will have trade-offs and of course Rust being well-designed will be better than Julia at some points.

37 Likes

I understand what you mean. But I wanted to have small examples. I am aware that the examples are trivial, I want people to focus on the effects, not on the problem itself and its implementation.

Personally, I would not read a blog post that has an example with more than 10 lines, unless it is a tutorial which this post is not.

I know that Julia is awesome for math problems. I use it regularly for my math and physics lectures. We both agree that it is much better than Python and C++.

I did solve real scientific problems both in Julia and in Rust. I did write the code for my bachelor’s degree in Julia (about 3700 lines of code). And I did physics projects in Rust (about 2300 lines of code). Plus I teach Julia and am preparing to a course for Rust. I think that I have a good idea about the tradeoffs of both languages and would much rather like to discuss concrete points than proving that I am allowed to do a proper comparison :slight_smile:

Here is where I got the plan for 2.0 from: 2.0 Milestone · GitHub
It says “Major changes and enhancements targeted for the next major release

5 Likes

I am saying that the material looks like cherry-picking because it uses some selected arguments that are expected to work in Rust’s favor simply because Rust is a static language. Does Rust offer more type-safety than Julia? Obviously - all together with F#, Haskell, C#, and many other languages.

It is known that the compiler will catch more bugs in static languages. This cannot be used as an argument for “Julia does not solve the two languages problem”.

Also, there is an argument where the material compares two different versions of pop! function.

A closer comparison to the Rust pop! version would be this one:

function rustpop!(x::Vector)
    isempty(x) && return nothing
    pop!(x) |> Some
end

function wrongusage()
    x = [1]
    rustpop!(x) * rustpop!(x)
end

Now, if you are aware of the Julia ecosystem, you might use JET - which will throw the following - without needing to run the code and encounter the error at runtime:

no matching method found `*(::Nothing, ::Nothing)`, `*(::Some{Int64}, ::Nothing)`, `*(::Nothing, ::Some{Int64})`, `*(::Some{Int64}, ::Some{Int64})` (4/4 union split): (UniLM.rustpop!(x)::Union{Nothing, Some{Int64}} UniLM.:* UniLM.rustpop!(x)::Union{Nothing, Some{Int64}})

Also, there is a good chunk in the blog post showing how to handle enum in Rust properly:

let mut v = vec![1.0];

let v1 = match v.pop() {
    Some(value) => value,
    None => 1.0,
};

let v2 = match v.pop() {
    Some(value) => value,
    None => 1.0,
};

v1 * v2 

# the above shows how pattern matching works in Rust,
# the actual production code would be reduced to:
v.pop().unwrap_or(1.0) * v.pop().unwrap_or(1.0)

Why not present the Julia equivalent? It will result in the same level of type-safety:

x = [10.0]

v1 = something(rustpop!(x), 1.0)
v2 = something(rustpop!(x), 1.0)

v1 * v2

Also - presenting an example like the usage of Vector{Any} as some performance foot gun is not a good idea: Vector{Any} can work wonders in part of the code where the performance is irrelevant (thus Julia’s flexibility). People will not use Vector{Any} inside some tight loop where the performance is crucial: and instead of writing v1 = [], they would use v1 = Int[] (or whatever type is appropriate in the context).

Some of Julia’s features can indeed be painful points when used in some sub-optimal way because you can switch between a Python-like mask and a more restricted, type-driven mode. And this is an argument favoring the claim that Julia took important “steps towards solving the two-languages problem” - not something to be held against the language when compared with Rust.

I think everybody appreciates a good comparison - and many Julia developers are aware of Rust’s strengths - but I don’t see how this kind of comparison is helpful or even fair.

25 Likes

I think that we both agree on this point, but we have to define the two-language problem first. For me, the two-language problem exists when you do prototyping and rapid development in one language for its flexibility and then rewrite the project in another language that is better for performance and “project scalability”.

I did say in my post that Julia is perfect when you want interactivity and rapid feedback.
I did also show that Rust is a better fit for bigger projects.

My conclusion is not that Rust solves the two-language problem. Not at all. I say that Rust is a bad fit if you are for example doing a data analysis. You don’t want to recompile, care about details etc.

My conclusion is that at least for me, if I am prototyping or I know that the project is time limited for about a week (will not be a huge one) and my problem requires interactivity, then I will use Julia because it is a better fit. But I will choose Rust if the project is going to be a bigger one which I will work on for weeks and it might run for a long time.

The two-language problem is not solved in my understanding if you have to think about the project’s requirements to pick the fitting language.

1 Like

Why you say that Julia cannot match Rust performance? Particularly for numerical code, avoiding heap allocations by preallocation solves the GC cost. Is there any other fundamental reason for Julia not matching Rust performance?

The blog post seems to appeal to the performance footguns, but that’s not really fair, as it doesn’t make sense to compare ultimate performance of good vs. poorly written codes - even if one of the languages is better in guiding the coder to avoid bad practices. It is fair to say that it is easier (and possible) to write slow code in Julia. But not necessarily true that it is harder to write fast code.

I can’t say for Rust, but the dynamism and easy of use of Julia led me to write faster code in it than in Fortran, which also doesn’t let the coder shoot it’s foot. It is fair to say that a new user will probably write faster Fortran than Julia to start with, but not when tunning algorithms and doing micro-optimizations come into play.

28 Likes

I am not sure if I agree. In your case, you are just saying: “if the project will be large I will start with Rust already because it will be the best language for it when it is finished”. Ok, but Rust is not good for prototyping, so you are just skipping this step and doing it directly in the final language (like you could with C/C++ before Rust). The advantage of Julia is that you can easily prototype with Julia, and then, for the final code, you are not required to change language to get good enough performance. Oh, okay, maybe the finished system will be more maintainable with Rust, I completely agree. But this does not mean Julia does not solve the two-language problem; Julia excels at prototyping, and is not impracticable (like it is with Python, for example) to have Julia as the final language for the system. Because of this, in my book, Julia does solve the two-language problem. It is not about being the best alternative in each and every case, but being great at prototyping and not being unreasonable to keep for the final system.

28 Likes

The talk

from the same workshop was also quite interesting: they’re using Julia for prototyping because it’s easier to implement and debug the algorithm, and Rust for deployment, because it’s easier to use from Python/C/C++, because in the end all their users are in Python. Quoting from their slides at 3:46

  • Julia easier for debugging “math” problems.
  • Rust easier for debugging “code” problems.

From what they say, the first point is mainly due to the fact in Julia you can more easily and dynamically inspect objects to see what’s really going on in the math.

44 Likes

For the counting example and data races, it would be fairer if you also included a Julia package for correctly addressing the issue since you used rayon crate in the Rust example.

julia> using ThreadsX

julia> safe_count() = ThreadsX.sum(x->1, 1:10_000)
safe_count (generic function with 1 method)

julia> safe_count()
10000
22 Likes

A fair comparison of the two languages should include examples of what is easy in Julia but may be difficult or impossible in Rust.

For example, there are high performance applications where embedding a DSL in a language and dynamically analyzing and compiling code written in the DSL can yield huge performance gains.

This is trivial in Julia but in most statically typed languages is difficult or impossible. Is dynamic compilation even possible in Rust? This is an honest question since I don’t know much about the language.

My package FastDifferentiation.jl could not evaluate derivatives efficiently without dynamic compilation. I’ve thought about porting it to Rust but concluded this would not be possible after consulting an experienced Rust programmer.

Symbolics.jl also uses dynamic compilation to generate efficient executables from symbolic expressions. There are doubtless other examples of high-performance dynamic code generation in the Julia ecosystem.

If dynamic compilation is possible in Rust could you explain how to do it? It would be great if it works.

Your case will be stronger if you don’t use straw man examples such as this one

v = Vec{Int64}(undef, 3)

which you say causes reliability problems because of the undefined elements. But you could have done this instead:

v = Int64[]
sizehint!(v,3)

which creates an array with capacity 3 but no stored elements. Pushing elements (up to 3) does not cause allocation. No undefined values.

Since 1.9 latency is no longer an issue, for me at least. Before 1.9 I had to create sysimages to get acceptable startup times but not anymore.

I would be interested in learning how big an issue Rust compile time is. For small projects this doesn’t seem to be a problem but I have read that as project size increases compile times can become very long. Have you built large systems in Rust, say 20Kloc and up, systems with many crates? What has your experience been? Is the compile time problem overblown?

13 Likes

Regarding Julia 2.0, there is no real concrete plan for that to happen any time soon and there is plenty of non-breaking work to do before then. Within Julia 1.x, Julia is committed to backwards compatability, although there have been some known exceptions. For example, the default dynamic task scheduler is potentially an issue for old code. Packages such as Compat.jl also help with forwards compatability.

Don’t take my word for it. Here’s Stephan Karpinski on the topic.

Also above that post you can see Matt Bauman saying note to take too much from the 2.0 milestone marking in Github.

Julia might have an easier time at maintaining backwards compatability than Rust since generally Julia source code is the primary form that is distributed.

9 Likes

Rust’s static analysis capabilities make it especially suited for ensuring compat. For example, this program automatically detects semver violations, which cannot be done in Julia (2023).

4 Likes

I think this is worth clarifying at the top of your blogpost or in your upcoming talk, pretty much “two language problem isn’t really a problem for development.” It’ll definitely go against popular opinion, as far as I know people still very much gravitate toward interactive scripting languages for quicker development and simpler interfaces to performant libraries, they desire that it all could be done in one language, and people have made many big performant packages in “scripting” languages. I’d also suggest highlighting aspects of Rust that people generally agree makes development quicker, like fewer explicit type annotations than other statically typed languages and support for generic functions, though not like Julia with multimethods.

For the record, I think it’s good to develop or encourage other languages for scientific computing, or to have different opinions on what the “problems” are with how it’s done and how to address them.

3 Likes

Years ago I worked at a supercomputer centre. The users were a wild bunch. One, a professor of quantum chemistry, humourously maintained that we shouldn’t put much effort into maintaining commercial packages like Gaussian because “if you use a commercial package, it’s not science”.

We didn’t follow his advice, but he had a point. Computational sciences at the edge develop their own software. Our users typically did. The methods are new, and will not be commercialized for years, if ever. It’s all prototypes, crafted together to compute stuff needed for a few publications. Then, if the methods are applicable to a wider set of problems, somebody comes along and develops a fortran or c+±library, an R package, a general system like GAMS, or field specific things like Gaussian, or some other reasonably well written general code.

I’ve used R for countless projects, most of them of no general interest, and most of them speeded up with embedded C or C+±code (which can be done inline in R with the Rcpp package). Some with fortran code linked in. Most scientists don’t do that, they are good at their own science, not at mixing languages, and most likely they only know one or two high level languages like matlab or R, and mindlessly follow best practice guidelines with no deeper understanding of the fine points of optimizations.

I believe rust is superb for professional programmers, putting together quality code for general use.

However, that’s not the typical scientist. That’s more of an exploratory activity, with a lot of trial and error, and changes being done, quickly. When it finally works, the coding is done, the results are written up for publication, the code is left to rot until someone asks for it. Using Julia ensures that you don’t have to enlist a programming specialist to get reasonable speed at each turn. In this sense, It does really solve the two language problem.

40 Likes

When googling for “Julia 2.0”, 4 out of 10 top hits I got were titled “Julia 2.0 isn’t coming anytime soon…” . Is it asking too much to make that much little research before making public statements on a scientific conference?

And no, “Major changes and enhancements targeted for the next major release” does not say there are any specific plans for that release.

9 Likes

It’s pretty nice that basically the only complaint for Julia here is the (obvious and totally expected) “static analysis is not as good as Rust’s”! All those error checking and interfaces are part of this story.

Given these:

You get arrays out of the box and LinearAlgebra.jl is preinstalled!

Julia also has awesome packages for solving differential equations, numerical integration and even newly symbolic calculation. Even dealing with units and measurement errors is a dream in Julia!

Julia is a much better fit for projects that …

  • use plotting

I would say Julia is a no-brainer for the vast majority of scientific software, if creating it anew and not updating an old existing system.
There are a lot of successful examples of projects much larger than

  • are time limited to about one week (a student’s submission for example)
12 Likes

People always associate dynamic with slow, but indeed that isn’t true. Using dynamism in order to delay optimization is definitely a strategy to optimize even more than purely static approaches, something Modelingtoolkit and FastDifferentiation both exemplify.

25 Likes

Sincere question here:

Suppose you have, deep inside your Rust code, a function f(inputs...), where inputs are some data structures which are produced by previous code.

Is it easy to isolate that to test, benchmark and optimize that function, independently of the rest of the code?

(in Julia one can dump the inputs, and then iteratively run the function with those inputs to fix/tune it. And, quickly, make of that a unit test).

Rust has so many good user interface tools that I would not be suprised there is a good way to do that, but that is one of the nice features of a dynamic language in terms of fixing algorithm errors and optimizing code.

4 Likes