Julia better rewritten in rust?

Offtopic
, ,
1

seeing the julia code is actually wrtten in c++, scheme, c and mainly julia

so comes a question to my mind
what if the c++/c code were rewritten in rust? or haskell
basically any memory-safe good-performance language?

2

What are you proposing would be the benefits?

2 Likes
3

I believe this is a good idea theoretically, but a very bad idea practically. Let me explain why:

Benefits of rewriting in Rust

Rust really is an amazing language. Not only is it blazing fast and memory safe as everyone keeps saying - it’s also

  • Easy to distribute, deploy and make cross-platform. Cargo is fantastic
  • Much higher level than C (maybe also C++, I don’t know much about that)
  • Idiomatic Rust has a low bug frequency, even of logical bugs - it’s thoroughly designed for safety and covering the edge cases.
  • The built-in compiler, linter and formatter are extremely helpful, capable and stringent - even if you’re a Rust beginner, using these tools feels like you’re pair programming with an expert.

However, the practical downsides of rewriting it in Rust are, unfortunately, more important

Disadvantages of rewriting

First of all, who is going to do the rewriting? Practically speaking, it’d have to be the people who are already working on the compiler, like Jameson and Kadowaki.

There are about 100,000 lines of C/C++ code in Julia. Maybe they could rewrite that in a year, if we’re optimistic. But should they spend a year of their schedule doing that? That’ll mean a year of potential Julia improvements missed just to re-implement what we already have. There are many exciting things on the horizon for Julia: Native code caching, static compilation, more heavy use of the effect system, and so on. Should all these be put on the backburner for very little gain?

Also - do these people know Rust already? Probably not - but they certainly know C. Should they spend months of their schedule getting good at Rust - on top of the year needed to rewrite the compiler?

Then there is the issue of bugs. Rust really is bug-resistant. But it’s not bug-immune. Rewriting any complex piece of code will introduce lots and lots of bugs, which have been painstakingly been hammered out of the current compiler after years of usage. Would the community accept reverting to a brand new untested compiler?

All taken together, a rewrite is an immensely large project which will put all other important improvements on hold, while also being dangerous in that it will make Julia more unstable by introducing bugs. And for what gain? Having the compiler in a somewhat nicer language that’s a little easier to maintain? It doesn’t make any sense.

34 Likes
4

On top of that, I imagine lots of the benefits rust has wouldn’t apply, since rewriting in rust would either entail a lot unsafe code, which defeats the point. Or having to rewrite large parts of julia itself.

That doesn’t mean it will never happen, and if someone wants to have fun, find a simpleish part of the C code in julia and rewrite it in Rust and then try to link it.

5

The reason Julia needs C++ is because LLVM, if there’s a LLVM in Rust maybe.

this is irrelevant, Julia is not implemented in C++

1 Like
6

There is LLVM in Rust because that’s how rust works :smile: so that wouldn’t be an issue

2 Likes
7

that’s ALSO how Julia works, Julia and Rust are parallel in the sense that both goes to LLVM IR before becoming machine code.

Swapping C++ LLVM for Rust LLVM buys you literally nothing, unless OP think there’s memory safety issue in the LLVM compiler, then maybe LLVM should be implemented in Rust not C++

2 Likes
8

One possibility here is that now that the compiler and runtime are separate, while the compiler does not have sense to be rewritten in Rust, maybe there is a benefit in writing the runtime in Rust, given that it is the one actually handling the memory of a Julia session, i.e. Array’s and GC.

Anyway, there is the problem of having yet another tool here, the rust compiler. And probably even handling the memory with Rust would need unsafe and would not make a giant difference to the status quo.

I also don’t know about how the IO is handled in the runtime, so surely I’m missing more things here.

2 Likes
9

All things may be made unsafe in the wrong hands.

2 Likes
11

that remembers me the wl-roots implementation for rust that at the end thanks to the rust’s memory management at compile time made it imposoble for how wl-roots manages memory giving unsafe rust code that at the end didn’t make any sence to continue the proyect

but then there is haskell that oviously in this proyect don’t make sence dubt to the haskell performance

12

i don’t now exactly how julia is implemented so i can’t say at all
but

if the c/c++ code of julia is used to implement base memory handling, and bitwise data representations
rust gives improvements by itself, on bug maintining related to the compiler itself, less memory errors on the garbage collector of julia at compile time, maybe jit compile time improvements, also could be lesser the julia redistribution size and since rust’s oop implementation is closer to funtional pg signifies more scalable code and safer

the downs of rewritten

are that the actuall implementation is stable enough and would be time consuming do it again and by the rust’s oop closer to haskell, so re-implement things from c++ is not that easy.
last but not least could have a bad impact on c/c++ code importation to julia, but improve the one of rust

13

Could you give a specific example of some code in Julia that is unsafe that you want addressed

14

unless you’re volunteering or commissioning someone to rewrite C++ LLVM into Rust LLVM, the Julia devs’ time better off spent on making Julia better.

Until you can find a concrete problem in C++ code in Julia that can’t be fixed easily or you find so many memory safety / bugs, it’s hard to say if it’s better at all.

Even as mission-critical and massive as Linux kernel hasn’t put switching to Rust as first priority.

7 Likes
15

For me, I would prefer to rewrite julia in julia.

Perhaps rewriting julia’s dependencies with rust would give the whole open source community more security.

1 Like
16

Is this kind of roadmap public somewhere? It all sounds extremely interesting.

17

Unfortunately not. There will likely be a “State of Julia” talk during JuliaCon, maybe the core devs will list out some goals for the next year.

Tim Holy is working on native caching, on and off. He hopes it’ll be in the 1.9 release. Shuhei Kadowaki and Keno Fischer is applying the effect system. There has been little concrete, visible work on static compilation, but the core devs had had it as a long-term goal for years.

6 Likes
19

This is the State of Julia talk given at JuliaCon 2022: JuliaCon 2022 Live Session Day 3 (Husain Attarwala, Relational AI, State of Julia, etc) - YouTube

5 Likes
20

Also this video on the precompile, codecaching and etc.

1 Like