# \[ANN\] DataSkimmer.jl - Summarize tabular data in the REPL

**URL:** <https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030>\
**Category:** Package Announcements\
**Tags:** tables\
**Created:** [March 26, 2021, 4:57pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030 "2021-03-26T16:57:47Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hasnep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hasnep/32/10912_2.png) [@Hasnep](https://discourse.julialang.org/u/Hasnep)\
**Post date:** [March 26, 2021, 4:57pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/1 "2021-03-26T16:57:47Z")

</div>

[DataSkimmer.jl](https://github.com/Hasnep/DataSkimmer.jl) exposes a function `skim()` which prints summary statistics in the REPL. It was inspired by the output of the [skimr](https://docs.ropensci.org/skimr/) R package.

The goal is to be able to summarise any `Tables.jl` compatible table, so if you want to help, try running `skim()` on any tables you have and send me examples where it breaks. 😅

Here’s an example using the iris dataset:

```julia
# Load some data
using RDatasets
iris = RDatasets.dataset("datasets", "iris")

# Skim the data
using DataSkimmer
skim(iris)

```

```plaintext
┌─────────────────────┬───────────┐
│ Type │ DataFrame │
│ N. rows │ 150 │
│ N. cols │ 5 │
│ N. numeric cols │ 4 │
│ N. categorical cols │ 1 │
│ N. datetime cols │ 0 │
└─────────────────────┴───────────┘

4 numeric columns
┌─────────────┬─────────┬──────────┬──────────┬──────┬──────┬──────┬──────┬──────┬───────┐
│ Name │ Type │ Missings │ Complete │ Mean │ Std. │ Min. │ Med. │ Max. │ Hist. │
├─────────────┼─────────┼──────────┼──────────┼──────┼──────┼──────┼──────┼──────┼───────┤
│ SepalLength │ Float64 │ 0 │ 100.0% │ 5.84 │ 0.83 │ 4.3 │ 5.8 │ 7.9 │ ▂▃▃▂▁ │
│ SepalWidth │ Float64 │ 0 │ 100.0% │ 3.06 │ 0.44 │ 2.0 │ 3.0 │ 4.4 │ ▁▃▄▂▁ │
│ PetalLength │ Float64 │ 0 │ 100.0% │ 3.76 │ 1.77 │ 1.0 │ 4.35 │ 6.9 │ ▃▁▂▃▁ │
│ PetalWidth │ Float64 │ 0 │ 100.0% │ 1.2 │ 0.76 │ 0.1 │ 1.3 │ 2.5 │ ▃▁▃▂▂ │
└─────────────┴─────────┴──────────┴──────────┴──────┴──────┴──────┴──────┴──────┴───────┘

1 categorical column
┌─────────┬────────────────────────────────┬──────────┬──────────┐
│ Name │ Type │ Missings │ Complete │
├─────────┼────────────────────────────────┼──────────┼──────────┤
│ Species │ CategoricalValue{String,UInt8} │ 0 │ 100.0% │
└─────────┴────────────────────────────────┴──────────┴──────────┘

No datetime columns

```

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [March 26, 2021, 5:06pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/2 "2021-03-26T17:06:31Z")

</div>

Looks cool. Any plans for customization?

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [March 27, 2021, 4:32am UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/3 "2021-03-27T04:32:17Z")

</div>

- Categorical summary could show the number of categories.
- “No datetime columns” seems unnecessary. There are lots of column types that _aren’t_ present.

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [March 27, 2021, 8:05am UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/4 "2021-03-27T08:05:12Z")

</div>

Would it be possible to support StructArrays v0.5? DataSkimmer currently causes some package downgrades - but I like it a lot!

---

<div class="post-metadata">

**Author:** ![Hasnep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hasnep/32/10912_2.png) [@Hasnep](https://discourse.julialang.org/u/Hasnep)\
**Post date:** [March 27, 2021, 11:54am UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/5 "2021-03-27T11:54:36Z")

</div>

I’d considered it, but I’m not sure what options would be useful, do you have any suggestions?

---

<div class="post-metadata">

**Author:** ![Hasnep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hasnep/32/10912_2.png) [@Hasnep](https://discourse.julialang.org/u/Hasnep)\
**Post date:** [March 27, 2021, 11:55am UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/6 "2021-03-27T11:55:50Z")

</div>

Thank you for your feedback, I agree with both of the points you made. The “No datetime columns” text was something I added when debugging and forgot to remove.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [March 27, 2021, 12:19pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/7 "2021-03-27T12:19:27Z")

</div>

@Hasnep, thank you for such a nice and extremely useful package.

Just a couple of comments, if you will:

- for the numerical entries, the number of digits printed could improve a bit further in terms of consistency? (see example below). Maybe a customizable parameter in a future version.
- The column “Missings” might be called “Missing”

 ![DataSkimmer_Excel_ex1](https://global.discourse-cdn.com/julialang/original/3X/4/9/49ce9e90d4a42e5386c8cccf51d1624e92e7e077.png)

In case you want to reproduce the above, the excel sheet was downloaded from Microsoft site as per working example below.

```julia
using XLSX, DataFrames
url = "https://download.microsoft.com/download/1/4/E/14EDED28-6C58-4055-A65C-23B4DA81C4DE/Financial%20Sample.xlsx"
download(url, "Sample_data2.xlsx")
df = DataFrame(XLSX.readtable("Sample_data2.xlsx", "Sheet1", infer_eltypes=true)...)

using DataSkimmer
skim(df)

```

_PS: do not know why discourse does not respect the REPL output formatting…_

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [March 27, 2021, 6:41pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/8 "2021-03-27T18:41:03Z")

</div>

It looks like it doesn’t work with `NamedTuple`s of `Vector`s or `Vector`s of `NamedTuples`s, the most “bare-bones” table types. Would you mind a PR to help implement that?

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [March 27, 2021, 7:28pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/9 "2021-03-27T19:28:55Z")

</div>

One can always convert those to a table/df first, and then skim them:

```julia
using DataSkimmer, DataFrames
nt = (a=1,b="hello") # named tuple
df = DataFrame((nt,))
skim(df)

```

 ![named_tuples_to_df_then_skimmed](https://global.discourse-cdn.com/julialang/original/3X/6/e/6e3784371472156d07a2c2d4d11273ce2aac447d.png)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [March 28, 2021, 7:09am UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/10 "2021-03-28T07:09:11Z")

</div>

> [@rafael.guerra](#):
>
> One can always convert those to a table/df first, and then skim them

Sure, but the whole point of

> **[GitHub - JuliaData/Tables.jl: An interface for tables in Julia](https://github.com/JuliaData/Tables.jl)**
>
> An interface for tables in Julia

is to make these conversions unnecessary.

---

<div class="post-metadata">

**Author:** ![Hasnep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hasnep/32/10912_2.png) [@Hasnep](https://discourse.julialang.org/u/Hasnep)\
**Post date:** [March 28, 2021, 7:01pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/11 "2021-03-28T19:01:12Z")

</div>

@oschulz

> Would it be possible to support StructArrays v0.5?

@rafael.guerra

> for the numerical entries, the number of digits printed could improve a bit further in terms of consistency? (see example below). Maybe a customizable parameter in a future version.

I have released version 0.2.0 which should address both of these. Thanks for your suggestions

@pdeffebach

> Would you mind a PR to help implement that?

I’d love a contribution! I started a branch [here](https://github.com/Hasnep/DataSkimmer.jl/tree/feature/accept-named-tuples) where I added some unit tests for those cases. You can use that as a starting point for a PR or make your own.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [March 28, 2021, 7:26pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/12 "2021-03-28T19:26:25Z")

</div>

@Hasnep, thank you.  
In the new version the above numeric data table looks prettier now:

 ![DataSkimmer_Excel_ex2](https://global.discourse-cdn.com/julialang/original/3X/5/7/57558ff4be89f0e3d4daf52987f09f1373cba331.png)

---

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [April 12, 2021, 9:54pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/13 "2021-04-12T21:54:40Z")

</div>

Cool package! `TablesSkimmer` may be a better name though.

---

<div class="post-metadata">

**Author:** ![Hasnep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hasnep/32/10912_2.png) [@Hasnep](https://discourse.julialang.org/u/Hasnep)\
**Post date:** [April 18, 2021, 8:29pm UTC](https://discourse.julialang.org/t/ann-dataskimmer-jl-summarize-tabular-data-in-the-repl/58030/14 "2021-04-18T20:29:37Z")

</div>

> [@matthieu](#):
>
> Cool package! `TablesSkimmer` may be a better name though.

I come from an R background, so I was thinking in terms of dataframes when I named it. That’s a better name, but my impression is that renaming packages in Julia is a bit inconvenient. 😕