Julia programmers use specializations everywhere while Rust’s type system does not support specializations: 1210-impl-specialization - The Rust RFC Book I’m curious to know about the impacts of this on the difference in the ecosystems.
1 Like
Yea, this and the dynamicity of Julia and other stuff are why this is harder than just translating rust’s traits verbatim… but I’m wondering if this is a fundamental gulf or is it fruitful to explore the safety vs flexibility tradeoff, especially as an opt in thing.
I don’t think dynamism is an issue here. In fact, the aforementioned RFC seems to suggest a reasonable solution (but I’m no Rust expert). Also, the RFC spends a lot of space discussing how to statically resolve what we call method ambiguities.
Yea, this and the dynamicity of Julia and other stuff are why this is harder than just translating rust’s traits verbatim… but I’m wondering if this is a fundamental gulf or is it fruitful to explore the safety vs flexibility tradeoff, especially as an opt in thing.
Got it. I was thrown off by the fact that it was never implemented.
For anyone following along here, we’re having a mega trait discussion about a potential implementation in base here: https://julialang.zulipchat.com/#narrow/stream/137791-general/topic/Trait.20system.20design.2C.20.20usecases.20.20and.20desiderata
and on slack data
You might want to reconsider your assumption that Rust’s memory safety makes it slow. It’s one of the fastest languages available, partly because of, rather than in spite of, the memory safety. See these benchmarks: https://github.com/drujensen/fib. What Rust runs in 7.652 s, Julia runs in 11.318 s.
I’m not sure about Julia, but Rust’s pyo3 package (crate) makes it really easy to expose Rust functions, structs (classes), and methods to python. I’d be shocked if there’s not something similar for julia.
2 Likes
While I agree that Rust is certainly not a slow language, and it’s memory model can sometimes be quite helpful for that performance, I don’t think a benchmark on the recursive Fibonacci sequence is really that great a demonstration of this point.
7 Likes
Well, one of the main objective of Julia is to solve the two-language problem and provide scientist with fast and productive language.
With this in mind, exposing Julia function to Python is less important than exposing rust function to Python.
1 Like
Especially since the fib microbenchmark doesn’t involve arrays at all…
The main problem with Rust right now is that, as far as I know, there is no support for parallelism con clusters. Even the MPI wrapper crate has had no commits in 4 years. I would love to try Rust on my project but unfortunately I need it to run on a cluster.
2 Likes
I think, this comparison is unfaire. You are comparing library not language itself. I believe there are handful of rust library can do this as simple as julia.
Regarding to two language problem resolver. In my opinion I think Rust is better suited here. As there provide similar abstraction. But rust is GC free. Which make it capable to write super fast program, this is the reason why it suitable for system program development.
Here is the computing model provided by rust & julia. (I also included c for comparison):
| | c | julia | rust |
| computation | v,o,c | v,o,c | v,o,c |
| | | | |
| abstraction | type | type(support abstract/union type) | type |
| | struct | struct | struct |
| | mutability | mutability | mutability |
| | scope | scope | scope(support ownership*) |
| | function(single method) | function(multiple method) | function(limited multiple method, support closure) |
| | macro | macro | macro |
| | | module | module |
| | | type generic | type generic |
| | | functor | functor(multi named, via impl) |
| | | | trait(interface + marker) |
| | | | safe |
| | | | |
* abstraction -> pattern, so template/automatic/check code can be generated by compiler.
* ownership can be seems as fine-grained scope control, which make function application as scope boundary. Rust use this restriction to generate memory management code in compiler
1 Like
Rust is a systems programming language with its core feature memory safety. It might be a very successful tool for systems programming and for writing safe and efficient code for huge libraries. For scientific computing applications, OTOH, a language without arrays support as first class citizens is a no-opt. If you say that I’m comparing the stdlib of Rust against Julia packages, no, Random is a stdlib in Julia.
Julia effectively solves the two-language problem, and now Rust re-invents what I call the library-language problem. As I said, Rust can be very successful for writing a large library for Python for example, instead of C++. But look at the insane productivity of Julia, one man in this awesome community, named Chris Rackauckas, wrote a whole ecosystem of Julia packages single handedly. This simply is not possible in other languages without a dedicated full-time team of programmers.
I like giving examples instead of much talk. Here is a 13-line Julia function that is 3X faster than its Rust equivalent besides being 3X shorter. I have to confess that I’m less versed in Rust than in Julia, so my Rust version might not be very Rust-y, but I struggled enough just to write the Rust version (any corrections are welcome of course). Look at how I compute the min and max of a vector in Rust, a very basic task, right? I Googled for many packages just to be able to do it.
Here is the Julia function and how much time it takes:
function histSum(x, y, bins=100)
bin_min, bin_max = minimum(x), maximum(x)
result = similar(y, bins)
α = bins / (bin_max - bin_min)
ab = α * bin_min
for (x, y) in zip(x, y)
i = min(bins, 1 + trunc(Int, muladd(α, x, -ab)))
result[i] += y
end
return range(bin_min, bin_max, bins+1), result
end
@btime histSum(x,y) setup = (x=rand(10_000); y=rand(10_000))
# 21.100 μs (1 allocation: 896 bytes)
and here is the Rust version:
extern crate rand;
use rand::Rng;
use itertools::izip;
use std::time::Instant;
fn hist_sum(x: &[f64], y: &[f64], bins: i64) -> ([f64;101],[f64;100]) {
// let bin = bins.unwrap_or(100);
let bin_min = x.iter().copied().fold(f64::NAN, f64::min);
let bin_max = x.iter().copied().fold(f64::NAN, f64::max);
let mut result = [0f64;100];
let alph = bins as f64 / (bin_max - bin_min);
for (i, j) in izip!(x, y) {
let ind = i64::min(bins-1, (alph * (i - bin_min)) as i64 );
result[ind as usize] += j;
}
let mut rnge = [0f64; 101];
for i in 0..101 {
rnge[i] = 1f64/alph * i as f64 + bin_min;
}
(rnge, result)
}
fn main () {
let mut x = [0f64; 10_000];
let mut y = [0f64; 10_000];
let mut rng = rand::thread_rng();
for i in 0..10_000 {
x[i] = rng.gen();
y[i] = rng.gen();
}
let now = Instant::now();
let c = hist_sum(&x, &y, 100);
let elapsed = now.elapsed();
let sec = elapsed.as_secs() as f64 + elapsed.subsec_nanos() as f64 / 1000_000_000.0;
println!("Micro-seconds: {}", sec*1000_000.0);
println!("{}", c.1[99]);
}
Finished release [optimized] target(s) in 0.71s
Running `D:\funest\target\release\funest.exe`
Micro-seconds: 60.099999999999994
51.02857714906178
8 Likes
There is not anything similar for Julia. On an absolute scale, people have done a lot of great work developing interop packages for Python. But, PyO3 (which is not just a package, but an org with several packages) receives far more concerted development effort. I’m not sure about the reason for this… using another dynamic language with a big runtime within Python is a very different approach. Big Python projects that need performance have several options. Rust is a good one. Julia is much harder to sell because it doesn’t have the same level of support. In this situation, the fact that you have to reimplement so many things that are already in Julia doesn’t matter. Because the Python interface is not a big priority in the community, Julia is out of the running.
I have a strong feeling that a serious improvement in Julia / Python interop would be a big driver in adoption. But, I don’t have the data to back this up.
2 Likes
I like julia syntax, and surely less words need to written, as: 1, no need to give memory management instruction/meta. 2, api is designed for high-level numerical computation.
I try to highlight that these 2 languages provide similar computing model level abstraction. So I don’t think the code amount will be a big difference between them.
Regarding to performance. You can not give a proof given one example, which even didn’t check the inside implementation and possible optimizations.
But I need to high light that GC in julia vs GC free in rust.
GC free language should capable of better performance in theory.
In theory, for most scientific computing algorithms GC won’t be much of an issue as one typically pre-allocates the arrays so there is fixed memory usage.
7 Likes
Is this a safe generalization? What about computations involving sparse matrices whose density isn’t known until runtime? Or dynamic programs using recursive functions and memoization where the number of indices to be evaluated depends on the problem data? Or plotting…
I’m not saying that only Julia can do these things, only that my understanding of “scientific computing” encompasses much more than in-place matrix/vector operations.
9 Likes
To me scientific computing usually implies solving differential equations. And GC is only an issue if one is constantly allocating and deallocating memory (so a sparse matrix that is dynamically grown is fine).
The only obvious case to me where this might be an issue is adaptive grid refinement, but meshing is so expensive that I don’t see memory allocation being the issue here.
1 Like
This is a common misconception but it’s generally acknowledged in programming language implementation circles that GC’d languages can achieve as good or better computational throughout by aggregating memory management work instead of interleaving it throughout program execution as non-GC languages do. Just because it’s manual/static, doesn’t mean that memory management is free in non-GC languages. You still need to find available memory when you allocate and make memory available again when it is no longer used. In C this is done by malloc and free calls and a malloc library that implements these. What non-GC languages do manage to avoid is the work required to discover reclaimable objects, since this is either determined by the programmer when they write the code or by the compiler when the code is compiled. But the language runtime still needs to do work to reclaim the memory and make it available for future use. And they end up spreading that work out in small bits and pieces. GC’d languages tend to batch the work up into larger chunks, which can make the work faster and potentially lead to better decisions. There are also performance-enhancing techniques that memory managed languages can use like compaction (related objects end up close together in memory), which can improve performance compared to what static memory manangemt can achieve (objects stay where they’re originally located). Julia doesn’t currently do compaction, but it could potentially in the future (but interacting with C makes this tricky). By no means am I saying that it’s easy to make a GC as fast as say C or Rust, or that Julia is there, but it’s not impossible.
The actual trade offs that GC imposes, rather than being necessarily slower, are:
- High-throughput GCs tend to have long pauses so they are inappropriate for applications with real-time requirements. Some exciting recent GC research has promise to deliver best of both worlds behavior here though, at the cost of substantial implementation complexity.
- Needing extra “work space” memory for the GC to work with. For server applications you want your working memory to be < 60% of the available memory for the process. For numerical applications where big arrays that don’t need to be copied or moved are likely to be a significant part of memory usage, the percentage is probably much less, but you still need some “room to maneuver”.
37 Likes
It’s worth noting that languages without GC may end up with more wasted memory than GC languages since they tend to hold on to objects for longer than they are being used (and have fragmentation issues)
3 Likes