# What's the current (spring 2024) canonical approach to data science in Julia?

**URL:** <https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [April 2, 2024, 3:11pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427 "2024-04-02T15:11:38Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [April 3, 2024, 4:28am UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/21 "2024-04-03T04:28:09Z")

</div>

> [@alfaromartino](#):
>
> `Tidier` → A wrapper of the popular R package. Notice that this is relatively new to the Julia ecosystem. I don’t think it still reached the point of considering “standard”.

It is not a wrapper around the R package. It is a reimplementation of the R TidyVerse in Julia:

> **[GitHub - TidierOrg/Tidier.jl: Meta-package for data analysis in Julia,...](https://github.com/TidierOrg/Tidier.jl)**
>
> Meta-package for data analysis in Julia, modeled after the R tidyverse. - TidierOrg/Tidier.jl

---

<div class="post-metadata">

**Author:** ![rdavis120](https://avatars.discourse-cdn.com/v4/letter/r/b5a626/32.png) [@rdavis120](https://discourse.julialang.org/u/rdavis120)\
**Post date:** [April 3, 2024, 12:57pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/22 "2024-04-03T12:57:26Z")

</div>

You could use duckdb for data analytics and it would make it easy to switch between languages.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [April 3, 2024, 2:04pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/23 "2024-04-03T14:04:24Z")

</div>

> [@nilshg](#):
>
> I’ve been trying to transition to Makie for years but in practice nearly all data I use has a time dimension and the fact that Makie doesn’t have date support so I can’t just do `scatter(df.date, df.price)` or whatever turns out to be severely limiting

That will probably be a thing of the past in 0.21. With [Unit support for Axes & Recipes, a.k.a axis converts by SimonDanisch · Pull Request #3226 · MakieOrg/Makie.jl · GitHub](https://github.com/MakieOrg/Makie.jl/pull/3226) :

```julia
using CairoMakie, Dates

lines(now() .+ Millisecond.(1:100), cumsum(randn(100)))

```

(The ticks aren’t great but that’s a different issue)

 ![grafik](https://global.discourse-cdn.com/julialang/original/3X/c/0/c03eb5ee523cbae25166e893f5a2a85b5196b4c2.png)

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [April 3, 2024, 2:12pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/24 "2024-04-03T14:12:49Z")

</div>

🤯

---

<div class="post-metadata">

**Author:** ![artemsolod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/artemsolod/32/20704_2.png) [@artemsolod](https://discourse.julialang.org/u/artemsolod)\
**Post date:** [April 3, 2024, 2:33pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/25 "2024-04-03T14:33:26Z")

</div>

Personally, I use `ggplot2` for plotting in jupyter notebooks via [GitHub - JuliaInterop/RCall.jl: Call R from Julia](https://github.com/JuliaInterop/RCall.jl)

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [April 3, 2024, 9:02pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/26 "2024-04-03T21:02:46Z")

</div>

Also note that unlike Python and others, you don’t really need any fancy specialized data structures for tabular data. So, using stuff like dataframes is completely optional!

For basic tabular operations, you don’t need any dependencies at all: vector-of-namedtuples is already a table, functions like `map` and `filter` are present in Julia Base.  
Such a table is fully-featured, convenient to work with, can be read/written to a file with any tabular IO packages (eg csv).

For more advanced operations, like groups or joins, there are packages to help. See SplitApplyCombine.jl (older & more popular), DataManipulation.jl (newer, faster, more extensive), FlexiJoins.jl (the most versatile join function to my knowledge, not just among Julia alternatives).

As a bonus, these data structures and functions take you much further than flat tables, with the exact same familiar interface.

---

<div class="post-metadata">

**Author:** ![berjine](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/berjine/32/51764_2.png) [@berjine](https://discourse.julialang.org/u/berjine)\
**Post date:** [April 4, 2024, 1:25pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/27 "2024-04-04T13:25:58Z")

</div>

I just want to add [InteractiveViz.jl](https://github.com/org-arl/InteractiveViz.jl?tab=readme-ov-file#interactivevizjl) which is based on Makie and is very useful when you want to visualize a large number of points

---

<div class="post-metadata">

**Author:** ![jonalm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonalm/32/16038_2.png) [@jonalm](https://discourse.julialang.org/u/jonalm)\
**Post date:** [April 5, 2024, 8:21am UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/28 "2024-04-05T08:21:40Z")

</div>

In addition to `DataFrames.jl` and `Makie.jl` as several people already have mentioned, I like to use [Transducers.jl](https://github.com/JuliaFolds2/Transducers.jl) for data wrangling. It takes some time to get used to, but makes it very easy to write reusable code for data transformations. I find the [MapCat](https://juliafolds2.github.io/Transducers.jl/dev/reference/manual/#Transducers.MapCat) transducers particularly useful.

---

<div class="post-metadata">

**Author:** ![cdawg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdawg/32/9811_2.png) [@cdawg](https://discourse.julialang.org/u/cdawg)\
**Post date:** [April 5, 2024, 8:41am UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/29 "2024-04-05T08:41:15Z")

</div>

Super exciting thread!

DataFrames is enough for me, also use RCall for missing Julia library functionality (like a well-tested batch correction algorithm!). Pluto + WGLMakie for data interaction, sometimes needs a reload, but is LOVELY when it works, you can figure out what your outliers are by mouse hover (DataInspector)!

Despite the downsides, Ive been using Pluto more and more because the reactivity means that once the script is right, you have a rock solid step in your pipeline with minimal statefulness. You MUST split off the expensive calculations and save the results to avoid locking on rerunning the big steps, but this is good hygiene anyway.

Id love to have a slightly more flexible reactivity and mutation, but Pluto is STILL worth it to me.

---

<div class="post-metadata">

**Author:** ![mihalybaci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mihalybaci/32/13528_2.png) [@mihalybaci](https://discourse.julialang.org/u/mihalybaci)\
**Post date:** [April 5, 2024, 12:59pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/30 "2024-04-05T12:59:03Z")

</div>

For everyone using Pluto and bummed about long running operations, it is possible to disable cells so that they don’t run with every change. Then when you need to, just enable the cell and reactivity will be turned back on. I’m sure many of you are aware of that already, but I thought I should mention it just in case.

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 5, 2024, 3:53pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/31 "2024-04-05T15:53:11Z")

</div>

I work in a large corporation using Julia daily for complex data manipulation daily. I use the usual `DataFrames.jl`, `Makie.jl`, etc for everyday tasks, but something perhaps unique is using [AlgebraicJulia/ACSets.jl: ACSets: Algebraic databases as in-memory data structures (github.com)](https://github.com/AlgebraicJulia/ACSets.jl) and sometimes the additional power of [AlgebraicJulia/Catlab.jl: A framework for applied category theory in the Julia language (github.com)](https://github.com/AlgebraicJulia/Catlab.jl) for manipulation of data that would otherwise be stored in lots of individual data frames. Being able to define a schema and algorithms on the schema level, knowing they will work with any particular data instantiations has been quite helpful.

---

<div class="post-metadata">

**Author:** ![kdpsingh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kdpsingh/32/215018_2.png) [@kdpsingh](https://discourse.julialang.org/u/kdpsingh)\
**Post date:** [April 5, 2024, 6:33pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/32 "2024-04-05T18:33:24Z")

</div>

As the author of Tidier.jl, I’ll just add that I generally agree with the comments above. However, we are aiming to make the package useful for Julia users at large and more than just former R users. I’ve personally used many other Julia and non-Julia frameworks and believe that tidyverse strikes a good balance between simplicity, functionality, and consistency across data analysis and plotting. Even though we have adopted some tidyverse defaults in Tidier, we allow for some of that to be tailored, and we take advantage of Julia functionality that couldn’t be as easily accomplished in the R version.

One of our current directions is to make Tidier code work on multiple backends through the TidierDB package. It will work with SQLite, PostgresSQL, DuckDB, and more. Fairly soon, the goal is to make it so that TidierData code will work nearly identically across data frames and databases.

So I wouldn’t write off Tidier as only geared towards R users, but I have used and benefited from DataFrames and DataFramesMeta, and Query.jl and understand the appeal of each.

Our README addresses some of this: [GitHub - TidierOrg/Tidier.jl: Meta-package for data analysis in Julia, modeled after the R tidyverse.](https://github.com/TidierOrg/Tidier.jl)

And I hope to put more work into our course consisting of Pluto notebooks.

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [April 5, 2024, 8:11pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/33 "2024-04-05T20:11:09Z")

</div>

Interesting. Do you have an example that you can share of how you solve a data science task with those tools?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [April 6, 2024, 11:22pm UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/34 "2024-04-06T23:22:23Z")

</div>

> [@slwu89](#):
>
> something perhaps unique is using [AlgebraicJulia/ACSets.jl: ACSets: Algebraic databases as in-memory data structures (github.com)](https://github.com/AlgebraicJulia/ACSets.jl) and sometimes the additional power of [AlgebraicJulia/Catlab.jl: A framework for applied category theory in the Julia language (github.com)](https://github.com/AlgebraicJulia/Catlab.jl) for manipulation of data

These sound interesting, but docs are quite involved. Do you happen to have simple examples showcasing this algebraic approach?

---

<div class="post-metadata">

**Author:** ![slwu89](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/slwu89/32/217323_2.png) [@slwu89](https://discourse.julialang.org/u/slwu89)\
**Post date:** [April 8, 2024, 4:30am UTC](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427/35 "2024-04-08T04:30:59Z")

</div>

@aplavin and @simsurace, yes, I’m trying to write a blog post after filtering out sensitive stuff. For now, it’s been very helpful to have essentially an in-memory database that I can store JuMP VariableRefs in to bring my data and optimization models together in a very natural way. Being able to migrate data between schemas is also essential for processing of output.

@aplavin I’d start with these 2 blog posts: [AlgebraicJulia blog - Graphs and C-sets I: What is a graph?](https://blog.algebraicjulia.org/post/2020/09/cset-graphs-1/) and then for some examples of conjunctive queries, [AlgebraicJulia blog - C-sets for data analysis: relational data and conjunctive queries](https://blog.algebraicjulia.org/post/2020/12/cset-conjunctive-queries/)

[Previous page](https://discourse.julialang.org/t/whats-the-current-spring-2024-canonical-approach-to-data-science-in-julia/112427.md?page=1)