The simplest linear fit with GLM

Tooling
9

Okay I’m not sure this forum is necessarily the best place to learn introductory regression modelling (for this I would recommend either “Introductory Econometrics” by Wooldrigde or Gelman et al’s latest “Regression and other stories”), but at least the zero one part should be straightforward:

Take the model

y_i = \alpha + \beta x_i

which includes a slope and intercept coefficient. You see that the second (slope) coefficient is multiplied by x_i, which is - when applied to a series of observations stacked in a vector - your x vector. The first (intercept) coefficient is multiplied (implicitly) by one - this is where the vector of ones comes from. In matrix notation if you have two observations y_1 and y_2, with corresponding covariates x_1 and x_2, your system of equations would be:

\begin{bmatrix}y_1 \\ y_2 \end{bmatrix} = \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta \end{bmatrix}

which if you multiply out the right hand side gives:

\begin{bmatrix}y_1 \\ y_2 \end{bmatrix} = \begin{bmatrix} \alpha + \beta x_1 \\ \alpha + \beta x_2 \end{bmatrix}

Hopefully that clarifies why the X matrix has a column of ones as the first column if you’re fitting a model with a constant.

I think GLM is consistent and well-documented in what it does, and see no reason for changing anything, but reasonably people can disagree on this. If you often find yourself using the non-@formula way of fitting models in GLM but want and intercept, you could just have the function in your PR as your own lm_with_intercept function.

3 Likes