# Correlation between categorial variables

**URL:** <https://discourse.julialang.org/t/correlation-between-categorial-variables/74411>\
**Category:** New to Julia\
**Tags:** statistics\
**Created:** [January 11, 2022, 5:06pm UTC](https://discourse.julialang.org/t/correlation-between-categorial-variables/74411 "2022-01-11T17:06:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![iskyd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iskyd/32/44797_2.png) [@iskyd](https://discourse.julialang.org/u/iskyd)\
**Post date:** [January 11, 2022, 5:06pm UTC](https://discourse.julialang.org/t/correlation-between-categorial-variables/74411/1 "2022-01-11T17:06:36Z")

</div>

I created a DataFrame from a CSV simply doing df = DataFrame(CSV.File(“input.csv”))  
Using describe(df) I noticed that I have both Int variables and String variables with eltype String7, String3 and so on.  
To calculate the correlation between int variables I simply do

```julia
num_var_names = names(df, Int64)[2:end]
num_vars = train[!, num_var_names]
cor_matrix = cor(Matrix(num_vars))

```

I want also to calculate the correlation between the categorical variables (identified as StringX).  
My first attempt was to simply do something like

`cor(Matrix(df[!, ["var1", "var2"]]))`

but I get  
MethodError: no method matching /(::InlineStrings.String7, ::Int64)

If I print out df[!, “var1”] I get  
`PooledArrays.PooledVector{InlineStrings.String7, UInt32, Vector{UInt32}}: [values...]`

I also try to convert to categorical doing

`cor(CategoricalArray(train[!, "MSZoning"]), CategoricalArray(train[!, "MSZoning"]))`

but i still get

MethodError: no method matching /(::CategoricalArrays.CategoricalValue{InlineStrings.String7, UInt32}, ::Int64)

So, is there a way to calculate the correlation between categorical variables? It would be good if I can do something as simple as I did for the numerical variables.

Thanks.

---

<div class="post-metadata">

**Author:** ![lawless-m](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lawless-m/32/30869_2.png) [@lawless-m](https://discourse.julialang.org/u/lawless-m)\
**Post date:** [January 12, 2022, 8:58am UTC](https://discourse.julialang.org/t/correlation-between-categorial-variables/74411/2 "2022-01-12T08:58:00Z")

</div>

If no-one comes forward with an existing solution, here’s how to do it in Python DataFrames

> **[Correlation between Categorical Variables](https://medium.com/@ritesh.110587/correlation-between-categorical-variables-63f6bd9bf2f7)**
>
> Correlation measures dependency/ association between two variables. It is a very crucial step in any model building process and also one…

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 12, 2022, 9:44am UTC](https://discourse.julialang.org/t/correlation-between-categorial-variables/74411/3 "2022-01-12T09:44:46Z")

</div>

As Matt’s link alludes to, this isn’t really a Julia but a methodology question:

```julia
help?> cor

(...)

  cor(x::AbstractVector, y::AbstractVector)

  Compute the Pearson correlation between the vectors x and y.

```

so `cor` gives you the Pearson correlation coefficient, defined as:

r = \frac{\sum (x\_i - \bar{x})(y\_i - \bar{y})}{\sum (x\_i - \bar{x})^2(y\_i - \bar{y})^2}

Clearly then `cor` doesn’t make sense for categorical variables, as there’s no notion of an average of categoricals, or the distance of an individual observation from another (or the mean).

The article above recommends the Chi-square test to measure dependency between two categorical variables, which is available in the `HypothesisTests` package here:

[https://juliastats.org/HypothesisTests.jl/stable/parametric/#Pearson-chi-squared-test-1](https://juliastats.org/HypothesisTests.jl/stable/parametric/#Pearson-chi-squared-test-1)

but there are other options (see e.g. an overview [here](https://juliastats.org/HypothesisTests.jl/stable/parametric/#Pearson-chi-squared-test-1) or a recent paper where a new test is proposed [here](https://arxiv.org/pdf/1811.11440.pdf)) so you might first want to decide which metric you’re interested in, then see if this is avaible somewhere in Julia.

---

<div class="post-metadata">

**Author:** ![iskyd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iskyd/32/44797_2.png) [@iskyd](https://discourse.julialang.org/u/iskyd)\
**Post date:** [January 14, 2022, 4:56pm UTC](https://discourse.julialang.org/t/correlation-between-categorial-variables/74411/4 "2022-01-14T16:56:10Z")

</div>

Thanks for the answer! I knew about chi2 test and methodology. I was just wondering if there is an easy way to calculate it directly from the dataframe, or a simple way to create the contingency table from the dataframe. The chi2 test is what i wanted to use.  
Thanks for the library and also thanks for the proposed paper, very interesting!