There seems to be quite some activity in the Python community about using pola.rs to get fast dataframes, with the aid of Rust. As a frequent DataFrames.jl user, I would be curious to hear @bkamins opinion on this “competitor”. Developer explaining the package pola.rs homepage [image]

That table is mostly meaningless. It only reflects that Polars attracts a wider audience, which is obvious since it is a package for both Rust and Python! I think DataFrames.jl alone is somewhat competitive with Polars, but less feature complete (by design?). DataFrames.jl in combination with some …

[image] tbeason: Polars has some cool built-in functions like a groupby_dynamic which would be nice to have. We are open to add any features that are doable in DataFrames.jl. If something is needed can you please open an issue to discuss how it would fit?

Yes, Polars seems excellent (and Pandas in some ways too, still more features, though I dislike much of its syntax at least in Python). There is a wrapper for Pandas, but not (yet) for Polars (I mean to the Rust code, you can call Python, that way you have indirect access; is the Python wrapper com…

[image] Palli: I don’t see any (inherent reason) for Polars to be better (faster, than Julia, or have more features) Inherent reasons: Polars is Arrow based only, that means it is less general, but can be optimized more for this specific case (strings is one of the major areas; note that Pa…

[image] bkamins: We are open to add any features that are doable in DataFrames.jl I don’t think that groupby_dynamic fits well within DataFrames.jl. It would be better for a time series or panel data package (right now I just do it manually in 2 steps).

[image] bkamins: Polars has lazy evaluation; DataFrames.jl does not have it and will likely not have it; tangent but also I asked on Slack already, it would really be great if something with Query.jl syntax can give us graph optimization (including making intermediate variables) before comput…

I absolutely love DataFrames.jl and I am very impressed with @bkamins constant efforts to make it better AND take the time to explain the features to a wider audience. Like in this thread for example. Looking forward to tomorrow’s blog post. Superblog

Pola.rs vs DataFrames.jl

General Usage Performance

bkamins February 23, 2023, 4:59pm 2

Polars is an excellent package that is actively developed and it has a decent performance.

In the comparison you shared there is an error, as DataFrames.jl has MIT License.

7 Likes

Topic		Replies	Views
Direct interface to Polars Rust library Data question	13	1880	November 9, 2023
DataFrames.jl data engineering performance compared with other softwares Performance performance	6	1043	November 10, 2021
A serious data start-up structured around a Julia data manipulation framework for larger-than-RAM data Offtopic	24	1046	September 16, 2024
JuliaDB, dataframes: Speculations over the future of data packages Data	24	7567	August 21, 2020
How much performance potential does DataFrames have? Offtopic question	7	4545	February 18, 2021

Pola.rs vs DataFrames.jl

Related topics