Welcome to the Julia community!
You can thank the virus and Ritchie Vink wanting to do it in Rust (people usually make software for selfish reasons in their favorite language):
the coronavirus has been in our country for a year, which means I have been sitting at home for a very long time. At the start of the pandemic, I had a few pet projects in Rust under my belt and I noticed that the “are we DataFrame yet”, wasn’t anywhere near my satisfaction. So I wondered if I could make a minimalistic crate that solved a specific use case of mine. But boy, did that get out of hand.
It’s unclear to me when Python was added, those seem to be the main supported languages, only ones with prominent install info. But I see in the docs R and JavaScript mentioned too.
Julia meaning DataFrames.jl used to compete with Polars, on H2Oai benchmark, but was usually a bit behind. InMemData.jl is a newer addition (and Tidier.jl building on the former).
It moved to here:
https://duckdblabs.github.io/db-benchmark/
From the top link:
5.2.2 Expensive locking
…
5.2.2 Lock-free hashing
Instead of the before mentioned approaches, Polars uses a lock-free hashing algorithm. This approach does do more work than the previous Expensive locking approach
Yes, single-threading is dead, but it’s unclear we are better at multi-threading than Rust, the reverse is likely true (note we are good at parallelism, myabe as good or better(?), and it’s strictly not the same thing, nor the same as concurrency). Rust and Pony are race-free languages, the latter an even better language seemingly, but Julia and e.g. Go aren’t race-free. Which means we need to be careful with locking, and such multi-threaded code is hard to do right.
It’s perfectly fine for us to rely on Rust in all areas we like, at least for Polars. Either depend on it directly or though its Python wrapper (does it have a better or more useful API?), by using PythonCall.jl.
Another reason, already mentioned, is the Python users, like precompipled code (usually C, some like it better than Rust, since Rust limits platform use, e.g. see crypto library, and the movement to Rust, e.g. FreeBSD people object if I recall).
Relying on Julia would limit platforms even more? Though we do support FreeBSD, i.e. the language, but NOT all of the ecosystem e.g juliacall/PythonCall.jl except through hacks (because of Mamba/Conda).
I still support:
I think we can learn from Pony (deny capabilities) and Rust and be excellent for multi-threaded with some language changes. Or even as POC with some package, and we already have some like that very recently GitHub - MilesCranmer/BorrowChecker.jl: A borrow checker for Julia
juliac is in 1.12, and if we had had it sooner, then it might have helped with adoption from other languages.