I am interested in the usual “does knowing X make you better at Y” theme,
where X=Rust and Y=Julia, as someone who more or less knows Julia (and basic C programming), but has no experience with Rust yet.
If you think that knowing Rust does not necessarily make you a better Julia programmer, what does this say about the design of Julia and the art or science of writing good/idiomatic Julia code (i.e. no allocations, runtime dispatch, cache invalidation).
When I have to code in rust, I clearly feel, that I am fighting the language. Every borrow makes me angry… Or is the language fighting me?
When I enjoy doing poetry in Julia it’s just a good time. I see solutions and reliefs.
That’s my experience emphasized.
From my own experience then yes, learning Rust did make me a better Julia programmer.
I think part of this is just that learning new programming languages that does things differently makes you a better programmer. You get to see which language constructs are basic (like loops and functions), and which vary a lot between languages (like the inheritance or object model). You get to see which code patterns work well in one language and poorly in another, and which aspects of languages and poorly and well designed.
With Rust in particular, it might be especially useful to know. I can’t tell because I don’t know a lot of languages, but:
So yeah, go for it!
Just out of curiosity, which APIs do you consider better in Rust and why?
Imho, especially multiple-dispatch can and has been used for some very clean APIs in Julia.
Several examples are in this blog post, especially vis a vis Rust. I agree with most (maybe all?) of it.
I didn’t find the borrow checker so burdensome in the end. You have to make an effort to absorb the rules. But I suppose your experience with the borrow checker depends strongly on the type of code you have to write. For some projects it may be more difficult.
I suspect that Julia has a lot of more applied like things in it and this has taken a lot of resources. For example Base has a lot of things to support technical computing and on low and high levels. My impression is Rust devs were more focused on designing a coherent, core, language. Julia developers also wanted to make an interactive tool for numerical computing. I didn’t do any kind of audit to back this up.
Ah ok, that’s were your coming from. Was thinking more about ease of use and composability of APIs where I find Julia often nicer than Rust.
In any case, adding to the original question of this thread: Learning another programming language is always good, especially if it requires/provides a different perspective on programming. Rust has a strong focus on correctness and builds on modern developments in type theory. Imho, it is more similar to Haskell than to Julia in this respect. Julia shows more of a Lisp heritage with dynamic typing, REPL driven workflows and multiple dispatch with implicitly defined interfaces (called protocols in Common Lisp). Overall, this can lead to quite different designs and not every best practice of Rust will work well in Julia (and vice-versa).
Learning how Rust handles traits and features via wrappers really changed how I think about project architecture in Julia. Rust and Julia both use composition as a core technique for building complex systems, and learning how to build systems that compose in Rust really enforces a lot of structure that I wish we had in Julia.
I think the borrow checker gets entirely too much focus when people discuss Rust. Instead, I’m a major proponent of Julia needing some sort of interface abstraction, in large part thanks to how Rust handles them.
I agree that the type system in Rust is nice. Yet, Julia is considerably more open in that every function is generic and can potentially be re-defined. Further, as data types are values even dispatch on specific values is possible and e.g., used to implement chain-rules or inverses for functions.
To me, a major difference is that Rust distinguishes between generic functions – which have a single implementation – and methods – with multiple definitions for different types (had discussed that difference previously in the context of Haskell). Thus, more thought is required up-front about what should be extensible and what not, which might be a good thing, but also restricts potential combinations/re-use when these had not been considered.
Some APIs that I think are better in Rust:
Path API, which we just don’t have in Julia. I think that’s a straight up design mistake on Julia’s side.Iterators sub-module: In Julia, you often have to do this:itr = get_my_itr(x)
itr = Iterators.map(foobar, itr)
itr = Iterators.filter(foofilter, itr)
itr = Iterators.drop(itr, 3)
# or alternatively
itr = Iterators.drop(Iterators.filter(foofilter, Iterators.map(foobar, get_my_itr(x))), 3)
Whereas in Rust:
let mut itr = get_my_iter(x).map(foobar).filter(foofilter).skip(3)
Note also how the dot-chaining makes the Rust iterator much easier to read (at least in my opinion).
There’s more stuff about iterators, of course: In Julia, it’s quite hard to implement something like Iterators.Stateful, for two reasons:
Stateful should store the state of the iterator it wraps, what type is that? It’s unknowable. This is because Julia requires you to handle the iteration state, while also claiming the iterator state is a private object.Rust’s slice type (e.g. &[u8]) is super basic and extremely useful. We don’t have something like this in Julia but we absolutely should have. And then we should implement major parts of Julia using that type.
In a similar vein, I like how Rust defines most of its string API on &str instead of String. That makes it much more consistent in avoiding string allocations compared to Julia, where the choice is typically either a slow, generic AbstractString fallback, and an optimised String method that requires allocating a new String.
Oh and another - I’ve been recently looking into Rust’s IO API compared to Julia’s unstructured collection of IO-related functions that we could call an “API”. Here they’re almost incomparable: Rust has a carefully documented set of 5 (!) methods that does nearly all the heavy lifting. Julia has a spawling, inconsistently designed mess that literally no-one understands.
What’s that for?
Operating on paths. IMO, we should have had @p_str defined in Julia, such that you should do open(p"/my/path") do ...
There are a bunch of code smell in Julia that arises from not having a path type. E.g.:
path in the name? Like joinpath, splitpath, abspath etc. Isn’t that a bit like if there were sumvector, mulvector, divvector etc? That to me suggests it should instead be join(::Path), split(::Path) etc.open("abc/def/ghi.txt") shouldn’t work on Windows. The separator / is just a convention - it really is the “same” path as abc\\def\\ghi.txt.AbstractString, then interpret it as a path, in a quite dubious manner. E.g. eachline(::String), which weirdly doesn’t iterate the lines of the string. Instead, it opens a file at the path given by the string and iterates lines from that file. Similar with copyuntil. In practise, that means you have to instantiate IOBuffers just for dispatch purposes, which is slow and inefficient.Apologies, this reply has gotten much longer than I originally intended!
TL;DR: Once you’re familiar with “regular” Rust, take a look at async and how it works under the hood. At the very least, the differences in usecases this was designed for should definitely further your understanding of general programming topics. So my take is that “knowing Rust” not only makes you a better Julia programmer, but a better programmer in general. Not because Rust is so much better than Julia, but because it exposes you to different paradigms.
One Rust API that’s probably pretty unknown to Julians that I think follows the same trend as the IO API mentioned above (in terms of being at least designed) is their async API (although I’m very sure that this is a controversial take 
).
Async functions in Rust don’t run at all until something actively polls them. In essence, the use of an async function in Rust is lowered to an implicit statemachine, where waiting on a task is equivalent to transitioning to another state in that statemachine. This allows custom so-called executors to be written that can make different decisions in how the tasks are run & scheduled (e.g. think of an async scheduler on an embedded system, which has very different requirements from one designed for web-request throughput!). This also means that this implicit statemachine can be (up to a point) manipulated to provide additional guarantees, e.g. by guaranteeing a specific (even reproducible) order of execution.
This design leads to the very curious result that Rust-async can compete with actual RTOS in terms of performance on embedded hardware! For some more information on how & why this is, see this blogpost:
The current Julia runtime/@async machinery can’t even begin to compete here, for a number of reasons that I won’t get into here so that this reply doesn’t grow more than it already has
Julia runs @async blocks eagerly and everything is hidden behind the actual Julia runtime, which means that the implicit statemachine can’t be manipulated and scheduling decisions depending on the blocking state of individual Task objects can’t be influenced. I tried writing such a Rust executor once for Julia, but ran into the problem that not everything that blocks a Task yields back into the scheduler (write on a non-stdout/stderr file doesn’t, for example). Not to mention that the Julia scheduler wouldn’t be aware of this either, which means that when it resumes a previously blocked task, the task wouldn’t then call back into my own scheduler for the actual scheduling decisions.
Further, composing async functions in Rust feels a bit easier than doing the same in Julia with @async/@spawn, where fetching the result of a Task is necessarily type unstable at the moment, which also means that composing tasks can often be much slower than necessary due to the forced dynamic dispatch.
Of course, that’s not to say that the async API in Rust is perfect - far from it. It’s certainly a difficult topic, and I’d wager most Rustaceans would agree that the current state of async is not at all satisfactory. Still, I’d like to think that even with all its warts, it’s a bit closer to “good API” than what we have at the moment. If you’re interested in what the current state of async in Rust is, this is a good in-depth summary:
Although you might want to wait with diving into this until you’re more familiar with Rust - quite a large part of the issues of async in Rust are due to the interactions with the borrow checker and lifetimes, so having a fairly good grasp on that is pretty much required to understand all of the nuances.
I would also add that we need AbstractPath to allow polymorphic implementations, which can be used with clouds and archives. I have also used AbstractPath to define the hashing of a directory at load and save serialising methods that reduces code duplication.
Thank you, that is very interesting and helpful.
Iterators are indeed a bit tricky in Julia and I usually reach for Transducers.jl instead. Imho, this is the right abstraction in that it decouples the actual traversal from the specification of desired operations. Also agree that pathnames could benefit from a proper abstraction (probably not super easy though, i.e., in Common Lisp it seems to be a running joke that someone always wants to fix pathnames there, but these come from a time with more diverse operating systems and various implementations of the language standard with slightly different interpretations).
I can do this:
julia> (1:10).map(x -> x + 1).filter(iseven).drop(2).collect()
3-element Vector{Int64}:
6
8
10
And it’s efficient:
julia> @btime (1:10).map(x -> x + 1).filter(iseven).drop(2);
0.991 ns (0 allocations: 0 bytes)
But it’s still only a curiosity. It’s not general or elegant to implement:
using DotCall
import Base.Iterators: Filter
import Base.Iterators: Drop
import Base.Iterators: drop as idrop
import Base: collect, Generator
function rmap(itr, f)
Iterators.map(f, itr)
end
function rfilter(itr, f)
Iterators.filter(f, itr)
end
@dotcallify UnitRange (map=rmap, collect)
@dotcallify Generator (map=rmap, collect, drop=idrop, filter=rfilter)
@dotcallify Filter (map=rmap, collect, drop=idrop, filter=rfilter)
@dotcallify Drop (map=rmap, collect, drop=idrop, filter=rfilter)
This is partly due to a few easy improvements that could be made to DotCall. (Eg, the macro can’t handle a non-Symbol name like Iterators.map.) But even then I doubt it would be very composable.
Could FilePathsBase.jl be of interest?
@jakobnissen and others have also mentioned the lack of composibility because it’s not adopted ecosystem-wide.
EDIT: There is a good discussion in this Discourse post: Better handling of pathnames
One comment asked why methods are not included for functions taking filenames in Base. I started from scratch following the example of Rust’s std::path with a type Path and PathBuf. It was relatively little work to add wrapper methods for many functions in Base (that for some reason are not wrapped in FileSystemBase). PathBuf is used for constructing paths and wraps IOBuffer. Rust’s std::path is more focused on efficiency and minimizing allocation. It may also be simpler in a way than FilePathsBase. There is no equivalent of a type hierarchy (which would be implemented with traits). It’s just two types each of which wraps a string. (Rust’s approach to mutability means it doesn’t need an IOBuffer for PathBuf.) A large amount of std::path wouldn’t make sense in Julia because of different language semantics. Still, I think it’s worth exploring. But I can’t invest much time at the moment.
My understanding is that the path API was inspired by the corresponding Python API at the time and never revised since, the problem is that Python got a much better API shortly after with a proper path type and we got stuck with the old design. Had that been designed a few months after we’d probably have had something different 