Hi, I’m working with a larger algorithm in which a recurring step (done 5 - 20 times) is to carry out a linear regression y^k = X\beta^k + \varepsilon^k with roughly 100.000 observations and 1.000-5.000 variables, where y^k is either a length 100.000 vector or 100.000 by 1.000 matrix that depends o…

[image] JADekker: As such, I’d like to save some computational effort and recycle some work if possible. The thing that comes to mind is to compute pinv(X) and to carry out the regression as pinv(X)*y^k. I was wondering if this is the way to go here, or if there is a better approach that I’m mis…

MWE: using BenchmarkTools, Random, LinearAlgebra Random.seed!(42) function test(N, K) X = rand(N, K) y = X * rand(K) + randn(N) @time X\y @time pinv(X)*y return nothing end test(200_000, 1500) gives 34.280021 seconds (9.01 k allocations: 2.238 GiB, 0.00% gc time) 34.37…

Try Xf = factorize(X) for k = 1:N beta = Xf\yk end the expensive part is the factorization of X, so you can perform this once only and then perform the relatively cheap Xf\y using the pre-factorized objct. if you’d like, you could choose the factorization manually, e.g., Xf = svd(X) Xf = qr…

Thank you both for your replies, I will do this. I wasn’t aware of the backwards-stability issue, thank you for pointing that out!

It turns out that after doing this, for any given problem that I consider, the vast majority of computational time now lies in three such QR factorizations that I need to perform. This is a major improvement, thanks a lot! I have access to a HPC. Suppose that the dimensions become a bit larger; wou…

[image] JADekker: I have access to a HPC. Suppose that the dimensions become a bit larger; would it still be advisable to just call qr(X, ColumnNorm()) or is there eventually some other way to make use of distributed/parallel computing that outperforms simply calling qr(X, ColumnNorm())? (If it …

Repeated linear least squares problem with large dimensions

General Usage Performance

Topic		Replies	Views
Pseudoinverse x Factorization in a repeated linear solve: when to use which? General Usage linearalgebra	6	211	October 6, 2024
Best way for linear regression problem on product features General Usage performance , regression , qr	28	521	April 11, 2025
Optimzing many linear solves Performance	2	298	June 14, 2023
Result changes depending on whether matrix is sparse or not General Usage linearalgebra	16	2656	November 4, 2019
How to do in-place least square `\`? General Usage linearalgebra	5	274	July 3, 2023

Repeated linear least squares problem with large dimensions

Related topics