Efficient way of doing linear regression

[Juan]

Related to your question:

GLM is slow on large datasets. Using OnlineStats for regressions? MixedModels?

Hello. I’m planning to move from R to Julia and doing some tests about how to properly deal with large datasets and do simple tasks like regressions or survival analysis. I’ve done a benchmark with R (microbenchmark) for the regressions. N ← 3000 x1 ← rep(1:N, N) x2 ← rep(1:N, each = N) x3 ← sqrt(rep(1:N^2)) x1x2 ← x1x2 gg ← rep(1:5, each=N^2/5) y ← 1-2x1+3x2+0.5x1x2+rnorm(N^2)+x3*rnorm(N^2) dat ← data.frame(y,x1,x2,x1x2,x3, gg) dat2 ← cbind(1,x1,x2,x1x2,x3,gg) lm(y ~ x1 + x2 + x1…

[chakravala]

If anyone is interested, i have implemented a faster and also numerically stable method for underdetermined linear systems in this post:

Grassmann.jl A\b 3x faster than Julia's StaticArrays.jl

This was because of a special explicit case for dimensions 1 and 2 and 3, which are now accounted for in Grassmann.jl also. Also, support has been added for Moore-Penrose inverses for underdetermined and overdetermined linear systems. For underdetermined cases, the exterior product algorithm works ~20x faster than the SMatrix algorithm, and it is numerically stable. For overdetermined equations, the method used is based on the traditional normal equations, and this is prone to more numerical i…

[lmiq]

Just to mention that I wrote a simple package to do what I was searching for when I found this thread, which was a very simple interface to do simple fits of 2D data:

https://github.com/m3g/EasyFit

[st1]

I also needed basic linear regression, a bit beyond X \ y, and didn’t want the full GLM dependency chain, so I tried to incorporate the various suggestions from this thread in a very simple, minimal-dependency package, see https://github.com/st--/LinearRegression.jl - in case anyone else who stumbles across this thread may find it useful!

I also put links to various packages for more fancy versions of linear regression (generalized, ridge, sparse, bayesian, online, …) into the readme.

[lawless-m]

great stuff, I just walked the same path