# Seeking advice on contributing to Julia: A faster Lasso solver

**URL:** <https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023>\
**Category:** Statistics\
**Tags:** question, package\
**Created:** [June 25, 2020, 3:30am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023 "2020-06-25T03:30:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![barankarakus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barankarakus/32/15938_2.png) [@barankarakus](https://discourse.julialang.org/u/barankarakus)\
**Post date:** [June 25, 2020, 3:30am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/1 "2020-06-25T03:30:13Z")

</div>

Hi there,

I’m a Master’s student in Statistics. I’ve recently been researching ‘safe screening rules’ for accelerating Lasso solvers. These can improve computation times by orders of magnitude when the number of features is large.

I’d like to create a package implementing these methods in Julia to create a faster Lasso solver than the standard co-ordinate descent algorithm. The reason for my post is that, while I’m a fairly experienced programmer and I (think I) know how to write good, maintainable code, I have never contributed to open-source projects before. I’m aware that writing code for use by others comes with a lot of baggage that writing code for oneself does not, so I’m looking for advice, be it general tips on open source development or Julia-specific advice.

Some specific questions I have in mind (apologies if any of these are overtly silly!):

- There’s a Lasso.jl package already, implementing the standard co-ordinate descent solver. Would it be best for me to implement my algorithm within this package - modelling the code style etc on this package - and submit a pull request? Or develop my own package, ‘FastLasso.jl’ or something?

- In the former case, how do I convince the maintainers of the package that my code and the the mathematics are sound? I could definitely write up a mathematical report of sorts - would that be expected approach? And I’m guessing they’d want to see applications of my solver and the existing solver to the same regression problem so that the two solutions can be verified to be the same (and computation times compared)?

Thanks!

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [June 25, 2020, 3:41am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/2 "2020-06-25T03:41:34Z")

</div>

Personally, I prefer fewer packages. But sometimes a PR is a drain on another maintainer’s time so it’s a difficult one.

I would write the code and keep it in a separate repo first, and then once i am happy with the result, I would write to the Lasso.jl maintainer to see if there’s apetite and bandwidth to incorporate it. If there is then I spend the effort to make it into PR. Otherwise I would make it into a separate pacakge.

I would also start the conversion with maintainer Lasso.jl as soon as I can. Preferably now to communicate the intention and options.

---

<div class="post-metadata">

**Author:** ![Albert\_Zevelev](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albert_zevelev/32/11844_2.png) [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Post date:** [June 25, 2020, 4:03am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/3 "2020-06-25T04:03:17Z")

</div>

I spent some time thinking about this.  
There are some areas in the Julia ecosystem that are pretty organized, where cooperation is easy. This doesn’t seem like one of those areas…yet

Here are some packages for penalized regression:  
[MLJ Linear Models](https://github.com/alan-turing-institute/MLJLinearModels.jl) @tlienart might be your best bet  
[Sparse Regression](https://github.com/joshday/SparseRegression.jl) @joshday  
[Lasso](https://github.com/JuliaStats/Lasso.jl).jl  
[Subset Selection](https://github.com/jeanpauphilet/SubsetSelection.jl)  
[GLMNet](https://github.com/JuliaStats/GLMNet.jl) (Fortran wrapper)  
[Orthogonal EM](https://github.com/jaredhuling/OrthogEM.jl) (unmaintained)  
[LARS](https://github.com/simonster/LARS.jl) (old)  
[Trimmed Lasso](https://github.com/copenhaver/trimmedlasso)  
[FISTA](https://github.com/klkeys/FISTA.jl), [IHT](https://github.com/klkeys/IHT.jl) (L1, MCP)  
[Constrained L1](https://github.com/Hua-Zhou/ConstrainedLasso.jl)  
[EmpiricalRisks](https://github.com/lindahua/EmpiricalRisks.jl).jl

I’m sure there are many others.

It breaks my heart to see beautiful code & effort go to waste when ppl retire from maintaining a package. It would be great if we made @tim.holy’s strategy part of our culture: when he was ready to step down from maintaining [Interpolations](https://github.com/JuliaMath/Interpolations.jl).jl he posted [Interpolations.jl needs a new maintainer](https://discourse.julialang.org/t/interpolations-jl-needs-a-new-maintainer/37454) & instead of letting [Interpolations](https://github.com/JuliaMath/Interpolations.jl).jl disappear into the abyss, another user offered to take over.

---

<div class="post-metadata">

**Author:** ![barankarakus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barankarakus/32/15938_2.png) [@barankarakus](https://discourse.julialang.org/u/barankarakus)\
**Post date:** [June 25, 2020, 4:16am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/4 "2020-06-25T04:16:15Z")

</div>

This is extremely helpful. Thank you.

---

<div class="post-metadata">

**Author:** ![barankarakus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barankarakus/32/15938_2.png) [@barankarakus](https://discourse.julialang.org/u/barankarakus)\
**Post date:** [June 25, 2020, 4:23am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/5 "2020-06-25T04:23:44Z")

</div>

Jeez, that’s a lot of packages for regression!

I actually tried searching for sparse regression solvers in Julia and the only packages I came across were Lasso.jl and glmnet.jl within the JuliaStats ecosystem. Either I’m awful at searching for Julia packages, or these packages are difficult to find for beginners to the Julia language (which I am), which isn’t ideal.

Out of curiosity, how did you know of these packages? Because you’re active within the Julia community or are they advertised in a nice fashion somewhere that I’ve completely missed?

---

<div class="post-metadata">

**Author:** ![Albert\_Zevelev](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albert_zevelev/32/11844_2.png) [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Post date:** [June 25, 2020, 4:31am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/6 "2020-06-25T04:31:31Z")

</div>

I copy and pasted these packages from a list I had prepared months ago when I was new to Julia & wanted to contribute to some of the ML packages.

There is a ton of great Julia stuff that is hard to discover.

Here is what I wrote about this in March:

> [@JuliaPDE: Survey of PDE Packages](https://discourse.julialang.org/t/juliapde-survey-of-pde-packages/35672/25):
>
> If I were searching for Julia repos on pdes, the following 6 are the ones I’d try:
> 
> 1 `#partial-differential-equations` 15 repos: [partial-differential-equations · GitHub Topics · GitHub](https://github.com/topics/partial-differential-equations?l=julia)  
> 2 `#pde` 11 repos: [https://github.com/topics/pde?l=julia](https://github.com/topics/pde?l=julia)  
> 3 `#pdes` 2 repos: [https://github.com/topics/pdes?l=julia](https://github.com/topics/pdes?l=julia)  
> 4 Observer: [https://juliaobserver.com/searches?utf8=✓&term=pde](https://juliaobserver.com/searches?utf8=%E2%9C%93&term=pde)  
> 5 Filter by tag: [https://pkg.julialang.org/docs/](https://pkg.julialang.org/docs/)  
> 6 Svaksha’s: [GitHub - svaksha/Julia.jl: Curated decibans of Julia programming language.](https://github.com/svaksha/Julia.jl)
> 
> Of course you wouldn’t discover repos that are not tagged or registered.  
> In none of these searches would you discover @jlchan’s repo on DG methods for pdes or several other nice repos in @PetrKryslUCSD’s survey.
> 
> The R community’s solution is [CRAN Task Views](https://cran.r-project.org/web/views/), for example see [Differential Equations](https://cran.r-project.org/web/views/DifferentialEquations.html).  
> Python is trying [Python Task Views](https://github.com/open-risk/pythontaskviews).
> 
> To be clear before [CRAN Task Views](https://cran.r-project.org/web/views/) R users have informally posted links to packages in their domains on their blogs etc. However, having something in an official place w/ volunteers who maintain it is more reliable.
> 
> There have been multiple requests for a Julia Task Views. I would be happy to volunteer to organize one on Data sources.

To repeat, MLJLinearModels.jl is likely your best bet, prob b/c it has great maintainers & is part of the AlanTuringInstitute.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 25, 2020, 8:46am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/7 "2020-06-25T08:46:30Z")

</div>

When implementing a specific, mostly standalone algorithm, I think it is best to put it in a small package, well-tested, documented, with a very lightweight interface.

This has the following advantages:

1. the code should remain functional work a long time, because of the simple setup, and should be easy to update when needed,
2. multiple packages can make use of it, possibly incorporating the algorithm into their own interface via a wrapper. All will benefit from improvements, and consequently the motivation to contribute increases.
3. if a particular dependent project is abandonned, your algorithm implementation will continue to be around (as opposed to having to be excavated from a complex and abandonned codebase).

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [June 25, 2020, 7:10pm UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/8 "2020-06-25T19:10:42Z")

</div>

all this being said… In my opinion all “penalized regression” can be thought of as Bayesian models with MAP estimators… instead of having lots of different ways of achieving that, it’s better to have a general purpose modeling language and some really good optimization routines… voila you have regression for free

---

<div class="post-metadata">

**Author:** ![barankarakus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barankarakus/32/15938_2.png) [@barankarakus](https://discourse.julialang.org/u/barankarakus)\
**Post date:** [July 4, 2020, 1:30am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/9 "2020-07-04T01:30:23Z")

</div>

This is great, thanks for your input. You’ve convinced me to go down this route!

---

<div class="post-metadata">

**Author:** ![Albert\_Zevelev](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albert_zevelev/32/11844_2.png) [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Post date:** [February 4, 2021, 7:02am UTC](https://discourse.julialang.org/t/seeking-advice-on-contributing-to-julia-a-faster-lasso-solver/42023/10 "2021-02-04T07:02:31Z")

</div>

@barankarakus were you able to complete your faster Lasso solver?