# Hello, I am stuck with a problem regarding Linear Regression and Ridge Regression

**URL:** <https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382>\
**Category:** New to Julia\
**Tags:** question, package, dataframes, regression\
**Created:** [January 28, 2022, 4:45pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382 "2022-01-28T16:45:16Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 4:45pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/1 "2022-01-28T16:45:16Z")

</div>

The following labeled data is given: x={10.9, 12.4, 13.5, 14.6, 14.8, 16.5, 19.4} and y={24.8, 30.0, 31.0, 29.3, 35.9, 39.6, 40.7}.  
perform linear and ridge regression.

I used the following code:

using DataFrames, CSV  
using PyPlot  
using Plots  
using GLM

x = [10.9, 12.4, 13.5, 14.6, 14.8, 16.5, 19.4] #x-values  
y = [24.8, 30.0, 31.0, 29.3, 35.9, 39.6, 40.7] #Y-values

num\_tr\_ex = length(y); #96

data = DataFrame(X=x, Y=y)

linearRegressor = lm(@formula(Y ~ X), data)

println(linearRegressor)

linearFit = predict(linearRegressor)  
(plot!(x, linearFit,))  
display(scatter!(x, y))

I have no idea how to get started with Ridge Regression.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 28, 2022, 6:37pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/2 "2022-01-28T18:37:58Z")

</div>

Is this a homework exercise?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [January 28, 2022, 7:34pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/3 "2022-01-28T19:34:08Z")

</div>

OP didn’t even bother to remove certain indicative phrases 😉

Ridge Regression is described [here](https://en.wikipedia.org/wiki/Ridge_regression) for example.

---

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 7:42pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/4 "2022-01-28T19:42:40Z")

</div>

Yes, But I did the code myself, I am currently stuck with ridge regression. Do you know any resources that might help?

---

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 7:43pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/5 "2022-01-28T19:43:45Z")

</div>

I didn’t quite understand? What do I need to remove?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [January 28, 2022, 7:45pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/6 "2022-01-28T19:45:04Z")

</div>

> [@fsanik041](#):
>
> Write the code to perform linear and ridge regression. Show R-squared error and adjusted R-squared error and explain results.

This reads like a homework task.

> [@fsanik041](#):
>
> Yes, But I did the code myself,

Fair enough. Maybe [Regression · Julia Packages](https://juliapackages.com/p/regression) could help?

---

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 7:47pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/7 "2022-01-28T19:47:49Z")

</div>

I’m looking for resources to help explain ridge regression, i know that in statistics, I just need to convert that into code.

---

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 7:49pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/8 "2022-01-28T19:49:13Z")

</div>

Thanks this link helps, but I wanted to look into a few examples and see how they did it to fully understand the coding part, I’m fairly new to Julia syntax. It would be great if you could help me with that!

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [January 28, 2022, 7:52pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/9 "2022-01-28T19:52:36Z")

</div>

> [@fsanik041](#):
>
> Thanks this link helps, but I wanted to look into a few examples and see how they did it to fully understand the coding part,

It looks to me like `GLM` doesn’t support ridge regression, see [Support for Ridge Regression · Issue #205 · JuliaStats/GLM.jl · GitHub](https://github.com/JuliaStats/GLM.jl/issues/205). I let others speak about the recommended replacement, because I’m not qualified…

Edit: a naive implementation could look like

```julia
function linear_regression(lambda, X, y)
    Xt = transpose(X)
    (Xt * X + lambda * I) \ Xt * y
end

```

---

<div class="post-metadata">

**Author:** ![fsanik041](https://avatars.discourse-cdn.com/v4/letter/f/a6a055/32.png) [@fsanik041](https://discourse.julialang.org/u/fsanik041)\
**Post date:** [January 28, 2022, 8:19pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/10 "2022-01-28T20:19:42Z")

</div>

Thanks a lot, I’ll look into it!

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [January 28, 2022, 9:14pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/11 "2022-01-28T21:14:44Z")

</div>

A ridge regression can be done as follows:

```julia
"""
    RidgeRegression(Y,X,λ,β₀=0)

Calculate ridge regression estimate with target vector β₀.
"""
function RidgeRegression(Y,X,λ,β₀=0)
    K = size(X,2)
    isa(β₀,Number) && (β₀=fill(β₀,K))
    b = (X'X+λ*I)\(X'Y+λ*β₀) #same as inv(X'X+λ*I)*(X'Y+λ*β₀)
    return b
end

```

This is from my [lecture notes](https://github.com/PaulSoderlind/FinancialEconometrics/blob/master/Ch07_OLS_Lasso.ipynb).

---

<div class="post-metadata">

**Author:** ![Eric](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eric/32/25565_2.png) [@Eric](https://discourse.julialang.org/u/Eric)\
**Post date:** [July 5, 2022, 9:44pm UTC](https://discourse.julialang.org/t/hello-i-am-stuck-with-a-problem-regarding-linear-regression-and-ridge-regression/75382/12 "2022-07-05T21:44:58Z")

</div>

An alternative is using the [LinearRegressionKit](https://github.com/ericqu/LinearRegressionKit.jl).  
There is an example [here](https://ericqu.github.io/LinearRegressionKit.jl/stable/ridge_regression_tutorial/) on how to use ridge regression.

Hereafter is a similar example with your data:

```julia
using LinearRegressionKit, StatsModels, DataFrames

using VegaLite

# Dataset

x = [10.9, 12.4, 13.5, 14.6, 14.8, 16.5, 19.4]

y = [24.8, 30.0, 31.0, 29.3, 35.9, 39.6, 40.7]

df = DataFrame(x=x, y=y)

f = @formula( y ~ x)

# traditional regression - as a baseline

lm = regress(f, df)

# ridge regression for some ks

rdf, ps = ridge(f, df, 0.0:0.005:1, traceplots=true)

rdf[1:5 , :]

# plot to see the impact of k on the coefs

ps["coefs traceplot"]

# plot to see the impact of k on the VIF

ps["vifs traceplot log"]

# the ridge regression for a selected k

rlm = ridge(f, df, 0.01)

```