Piecewise linear regression with automatic knot selection

Hi everyone,

I am new to Julia, mostly work with R. I am looking for the equivalent of the “segmented” package in R, to fit a piecewise linear (constant in my special case) regression where the knot location is automated. Does this functionality exist in Julia?

Hello and welcome to the community

Would EllZeroTrendFiltering.jl perhaps fit your bill? I do not think that this package fits the regression models for you, but it solves the most difficult problem, finding the optimal knot points by solving a dynamic-programming problem. With the knot points determined, it should be straightforward to use any of the standard packages for linear regression to fit the models.

2 Likes

Thank you @baggepinnen for a perfect warm welcome! That is super helpful, it got me thinking. I am concerned that the optimal knot locations that emerge for a piecewise constant function may be not be same as the knots for a piecewise linear function. I am wondering if the recursive problem in the piecewise constant case is identical to fitting a 1-D regression tree.

The package should find you a piecewise linear function, from the readme

We want to find a piecewise linear, continuous function

Related: If you first perform an ordinary regression which returns a function for evaluating your regression model, then the problem reduces to finding a piecewise linear approximation to that function. This thread contains several methods for performing adaptive piecewise linear interpolation of an arbitrary function to a user-specified error tolerance.

Quick question–is there a reason your function has to be piecewise-linear? Smooth approaches like LOESS are almost always better.

For what its worth, I made this package that explicitly does what you want: GitHub - stelmo/LinearSegmentation.jl: Linear segmentation

3 Likes

Could you explain the method and how to use it in the above context?

There is a simple method based on a proximal gradient method which is described by Adaptive Piecewise Polynomial Estimation via Trend Filtering.

I have some code for doing linear regressions where the knots are also estimated - all coded up as a NLS/GMM problem. See section 12.2 of these notes (I could probably share the code, if wanted.)

2 Likes