Below, I show some generated data for a linear regression. With these features (also known as variables, covariates or predictors)
A, B, C, D and
E, I aim to predict an outcome
Y. Would the data shown below be considered multicollinear in the sense that it could become problematic for linear regressions?
I would say yes, and I read that Bayesian models can handle collinear data well, so I expected a huge difference between a Bayesian and Frequentist model. However, I compared a Bayesian to a Frequentist model and they gave the same outcomes, see the figures below. Therefore, I concluded that the Frequentist model did not have any issues with the collinearity.
Might this be because
lm from GLM uses QR decomposition? Or, is my data not correlated enough? When would
GLM.lm start showing huge variances as mentioned in Wasserman’s lecture notes?