# Single variable least absolute deviation (LAD) linear regression

**URL:** <https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386>\
**Category:** General Usage\
**Created:** [March 2, 2019, 9:04am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386 "2019-03-02T09:04:36Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [March 2, 2019, 9:04am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/1 "2019-03-02T09:04:36Z")

</div>

I’m looking for a simple to use single variable least absolute deviation linear regression library. My problem is modeled by a single variable function with small set (less than 32) of observations. I want to get a linear approximation of it using least absolute deviation.

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 2, 2019, 9:24am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/2 "2019-03-02T09:24:10Z")

</div>

something like this perhaps?

```julia
(T,K) = (size(x,1),size(x,2))

  b_old = zeros(K)
  b = fill(1e+6,K) .+ prec
  wii = ones(T)
  while maximum(abs.(b - b_old)) > prec
    b_old = copy(b)
    xw = x.*wii
    b = (xw'x)\(xw'y) #solve (xw'x)b=xw'y for b
    u = y - x*b
    wii = min.(1.0./abs.(u),1/epsu) #Amemiya p 78, since 1/abs(u) can be infinite
  end

  yhat = x*b #fitted values
  u = y - yhat

  Sxx = x'x/T #Sum{xt)*x(t)'}/T
  h = 1.06*std(u)/T^0.2 #kernel density value at u=0
  Kh = exp.(-0.5*((u/h).^2))/sqrt(2*pi) #gaussian kernel, with "std" = 1
  f0 = mean(Kh)/h
  Covb = f0^(-2)*inv(Sxx)/(4*T) #Cov(b)

```

---

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [March 2, 2019, 9:54am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/3 "2019-03-02T09:54:32Z")

</div>

Thanks, I will give it a try. What are the dependencies?

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 2, 2019, 10:32am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/4 "2019-03-02T10:32:09Z")

</div>

you need to do `using Statistics` (for the `mean` and `std` calculations). It’s one of the standard libraries, so no installation is required.

Before running, the code you need to define `y` (vector with dependent variable), `x` (matrix of regressors), `prec` (perhaps 1e-5) and `epsu` (perhaps 1e-6)

---

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [March 2, 2019, 11:13am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/5 "2019-03-02T11:13:53Z")

</div>

What’s matrix of regressors? I’m not familiar with statistical lingo. Let’s say my observations are 1-\>3, 2-\>4, 3-\>7. Do I define:

```julia
x = [1, 2, 3]
y = [3, 4, 7]

```

How do I get predicted values?

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 2, 2019, 12:07pm UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/6 "2019-03-02T12:07:51Z")

</div>

The code wants `x` to be a matrix, so do `x = [1 2 3]'`. Then you find the fitted values in `yhat`

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [March 2, 2019, 3:25pm UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/7 "2019-03-02T15:25:12Z")

</div>

It may be overkill here, but is there a succinct way to do it with Flux?

---

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [March 2, 2019, 10:32pm UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/8 "2019-03-02T22:32:59Z")

</div>

> [@Paul\_Soderlind](#):
>

I have a bit of a problem with the results. For:

```julia
x = [1 2 3 4]'
y = [0 1 0 1]'

```

I’m getting `yhat`:

```julia
4×1 Array{Float64,2}:
 0.24999448434228871
 0.49998896868457743
 0.7499834530268661 
 0.9999779373691549

```

where `sum(abs.(yhat-y))` (LAD) is `1.5000110313154225`. But

```julia
y0 = [0 1/3 2/3 1]'

```

provides much better approximation with LAD of `1.3333333333333335`

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 3, 2019, 10:16am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/9 "2019-03-03T10:16:37Z")

</div>

> y0 = [0 1/3 2/3 1]’

is what you get when you also include a constant among the regressors. You get this by changing the matrix of regressors to

```julia
x = [1 1;
     1 2;
     1 3;
     1 4]

```

---

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [March 3, 2019, 11:19am UTC](https://discourse.julialang.org/t/single-variable-least-absolute-deviation-lad-linear-regression/21386/10 "2019-03-03T11:19:47Z")

</div>

Thank you. It works now.