Dear Julia friends, I am benchmarking Julia for linear regression models and trying to optimize that for high-dimensional spaces. Any suggestions on the code below to reduce allocations and speed up the code? Thank you so much using Random using GLM using DataFrames using BenchmarkTools function b…

This discussion was had before, much of it is likely unchanged: [image] Efficient way of doing linear regression Performance Hello, I need an efficient way to performs linear regression because I have to fit several segments in a big for loops which takes time to achieve…

I just profiled the code with @profview benchmark(1000) and on my computer, this call spent roughly 50% of the time in lm, which is split pretty evenly between ModelMatrix, ModelFrame and fit 30% of the time in DataFrame (constructing data) 20% of the time in randn (generating X) One would assum…

Have you considered just skipping the dataframe and GLM stuff and just doing fit = X \ y directly? That’s all it’s doing in the end. (Reducing it to X \ y lets you further optimize things by customizing the method of the linear solve, doing things in-place, etcetera. It may also trim off a bunch…

lm also calculates other values like t-statistics etc. It would be cool to have more control over the solver though.

You might want to try FixedEffectModels.jl , which also does linear regression but may be faster. Also, your benchmark includes creating the DataFrame. You should split the data creation and estimation process into separate pieces to best analyze the performance of the estimation.

Yes, I am actually calling it several times (like thousands), so any small improvement is critical

fit or X \ y complexity also scale differently from other operations (cubic in dimension, IIRC), and therefore testing benchmark(10_000) should spend most time there and optimizing the rest will be less crucial. Having said this, for smaller problems it is always nice to reduce overhead. One would…

Do you always have the same X? or the same y? Do they at least keep the same dimensions?

The R world has a nice separation between a small lm.fit() which is fast and the full lm() which does more things. We should think similarly. Mousum Dutta has been building a QR-decomposition based engine for GLM.jl which is more precise. Here also, we see the tradeoffs between a fast and precise m…

That would be great, we would have more control

Accelerating linear methods

Specific Domains Machine Learning

Leticia-maria June 3, 2023, 11:15pm 11

They have the same dimension: I have posted a more detailed code here: Error when processing data from file for linear regression

Topic		Replies	Views
GLM is slow on large datasets. Using OnlineStats for regressions? MixedModels? Performance glm	25	5319	November 26, 2018
Error when processing data from file for linear regression Performance question	7	344	June 5, 2023
Parallel/multithreading version of linear regression Performance performance , parallel , multithreading , distributed , regression	19	2131	October 28, 2020
Simulation of regression coefficients New to Julia first-steps , dataframes	7	623	June 29, 2021
Efficient way of doing linear regression Performance regression	44	21227	February 7, 2022

Accelerating linear methods

Related topics