# First impression of DataFrames.jl

**URL:** <https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [November 7, 2020, 8:45pm UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753 "2020-11-07T20:45:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![digitalpig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digitalpig/32/19037_2.png) [@digitalpig](https://discourse.julialang.org/u/digitalpig)\
**Post date:** [November 7, 2020, 8:45pm UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753/1 "2020-11-07T20:45:23Z")

</div>

Hi,

Trying to step into Julia these days, so far so good. Really excited about the multiple dispatching and type hint. Got a few questions regarding DataFrames.

Currently I am using the prostate cancer dataset from [here](https://web.stanford.edu/~hastie/ElemStatLearn/datasets/prostate.data). It is a 97x11 dataset. In pandas, you can call the `corr` method on the dataframe itself to generate a correlation matrix of all columns. Is there a similar method that I can call to get that on the Julia side?

Thank you!

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [November 7, 2020, 9:15pm UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753/2 "2020-11-07T21:15:43Z")

</div>

Hi! Thank you for using DataFrames.jl. What type and shape of output you would expect? If you want a `Matrix` then just do:

```julia
using Statistics
cor(Matrix(your_data_frame))

```

but things with calculating correlations are quite involved so depending on the details you want a specific answer might be different (in particular - do you have missing values and how do you want to handle them?).

EDIT: what type of correlation do you want to calculate (I assumed Pearson correlation coefficient)?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [November 8, 2020, 12:41am UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753/3 "2020-11-08T00:41:18Z")

</div>

> [@digitalpig](#):
>
> In pandas, you can call the `corr` method on the dataframe itself to generate a correlation matrix of all columns.

In [DataConvenience.jl there is `dfcor`](https://github.com/xiaodaigh/DataConvenience.jl)

`dfcor(df)` should work.

---

<div class="post-metadata">

**Author:** ![digitalpig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digitalpig/32/19037_2.png) [@digitalpig](https://discourse.julialang.org/u/digitalpig)\
**Post date:** [November 8, 2020, 12:51am UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753/4 "2020-11-08T00:51:09Z")

</div>

Hi! Thank you for the reply. I am looking for something like this:

 ![correlation](https://global.discourse-cdn.com/julialang/original/3X/9/9/99ea74377bc88eabe41b44b941002d17f27adb37.png)

And you are right, for now I am thinking of pearson correlation.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [November 8, 2020, 7:22am UTC](https://discourse.julialang.org/t/first-impression-of-dataframes-jl/49753/5 "2020-11-08T07:22:05Z")

</div>

You can get it like this:

```julia
julia> using DataFrames, NamedArrays, Statistics

julia> df = DataFrame(x1=rand(10), x2=rand(10), x3=rand(10), x4=rand(10))
10×4 DataFrame
│ Row │ x1 │ x2 │ x3 │ x4 │
│ │ Float64 │ Float64 │ Float64 │ Float64 │
├─────┼──────────┼──────────┼───────────┼────────────┤
│ 1 │ 0.158358 │ 0.942125 │ 0.713538 │ 0.630956 │
│ 2 │ 0.374393 │ 0.813797 │ 0.504182 │ 0.947029 │
│ 3 │ 0.520227 │ 0.542249 │ 0.833646 │ 0.609631 │
│ 4 │ 0.928563 │ 0.402397 │ 0.444998 │ 0.232351 │
│ 5 │ 0.230898 │ 0.824582 │ 0.199968 │ 0.00203982 │
│ 6 │ 0.197203 │ 0.84624 │ 0.408122 │ 0.636816 │
│ 7 │ 0.168241 │ 0.281407 │ 0.665497 │ 0.949534 │
│ 8 │ 0.494666 │ 0.39342 │ 0.236596 │ 0.522137 │
│ 9 │ 0.431282 │ 0.425107 │ 0.0946223 │ 0.584611 │
│ 10 │ 0.639034 │ 0.256714 │ 0.4461 │ 0.298986 │

julia> struct NoPrint end; Base.show(::IO, ::NoPrint) = nothing

julia> NamedArray([i > j ? cor(df[!, i], df[!, j]) : NoPrint() for i in 2:ncol(df), j in 1:ncol(df)-1],
                  (names(df)[2:end], names(df)[1:end-1]))
3×3 Named Array{Any,2}
A ╲ B │ x1 x2 x3
──────┼────────────────────────────────
x2 │ -0.555227                      
x3 │ -0.102989 0.0988691           
x4 │ -0.407655 0.0381506 0.444643

```

note again - that this assumes you do not need to do handling of `missing` values (as there are several strategies that could be used here).