Julia analog to R's poly function, orthogonal polynomials throught QR decomposition

Hey guys!
Im trying to replicate some existing R code in julia for a proyect, but such R code depends on the stats base poly function which, given a collection of points, it computes orthogonal polynomials evaluated at those points.

I havent been able to find an analgoue in Julia, so I’m trying to replicate the necessary functionalities of the poly function. Regardless, after checking the code, it uses a QR decomposition to obtain such evaluations. I’m trying to understand what is doing, but the only similar thing I´ve found on books is the utilization of the QR decomposition to compute OLS.

Does anybody know if there is an analogous function in Julia, or know of a reference which can be usefull to understand poly’s background work?

2 Likes

I had to search around a bit to understand exactly what kind of orthogonality R’s poly function refers to. It seems to be orthogonality with respect to the following inner product:

\langle p, q \rangle = \sum_{i = 1}^n p(x_i) q(x_i) \, ,

where the x_i are the points provided by the user. Let’s label this family of polynomials p_k(x), where k is the degree of the polynomial.

Since you’re only interested in the values p_k(x_i), rather than the coefficients defining the polynomials, what we’re actually trying to calculate is just a matrix Q defined as

Q = \begin{bmatrix} p_0(x_1) & p_1(x_1) & \cdots & p_m(x_1) \\ \vdots & \vdots & ~ & \vdots \\ p_0(x_n) & p_1(x_n) & \cdots & p_m(x_n) \end{bmatrix}

which should satisfy the following properties:

  1. The columns should be orthogonal (this is the orthogonality of the polynomials)
  2. The column space of Q should be the vector space spanned by polynomials of degree up to m evaluated on the x_i, which is mostly easily expressed as the column space of the Vandermonde matrix V:
V = \begin{bmatrix} 1 & x_1 & x_1^2 & \cdots & x_1^m \\ \vdots & \vdots & \vdots & ~ & \vdots \\ 1 & x_n & x_n^2 & \cdots & x_n^m \end{bmatrix}
  1. The first k columns of Q should have the same span as the first k columns of V (this ensures that p_k(x) is a polynomial of degree k)

These are exactly the properties satisfied by the factor Q in the “thin” QR decomposition of the Vandermonde matrix, V = Q R. Thus, what we need to do is a) form the Vandermonde matrix, and b) compute its QR decomposition. Each column of the resulting Q contains the values of one of the polynomials p_k(x). Here’s an implementation

using LinearAlgebra

function poly(x, degree)
    V = reduce(hcat, (x.^k for k in 0:degree))
    F = qr(V)
    return F.Q[:, 2:(degree + 1)]
end

Note one last detail: R’s poly doesn’t return the first column, which is just constant; to mirror this behavior, the implementation above drops the first column of Q and returns columns 2:(degree + 1).

Let’s verify that it works:

julia> poly(1:10, 3)
10×3 Matrix{Float64}:
 -0.495434    0.522233    0.453425
 -0.385337    0.174078   -0.151142
 -0.275241   -0.0870388  -0.377854
 -0.165145   -0.261116   -0.334671
 -0.0550482  -0.348155   -0.12955
  0.0550482  -0.348155    0.12955
  0.165145   -0.261116    0.334671
  0.275241   -0.0870388   0.377854
  0.385337    0.174078    0.151142
  0.495434    0.522233   -0.453425

The first two columns here match this StackOverflow answer demonstrating usage of R’s poly: https://stackoverflow.com/a/19484326, so I’d say this passes the sniff test.

10 Likes

Does anybody know the math well enough to see if anything in this package or the other linked packages are related at all? GitHub - jverzani/SpecialPolynomials.jl: Families of polynomials

1 Like

@Benny in short: orthogonal polynomials are orthogonal with respect to some inner products. These inner products can be defined by integrating the products of the polynomials against a distribution (see e.g. the Hermite polynomials for a normal distribution and the Laguerre polynomials for an exponential distribution); some such polynomials are implemented in the package you mentioned. The QR based method above instead enforces orthogonality “in sample”; i.e. with respect to the inner product defined above.

Tl;dr both create orthogonal polynomials, but they differ in whether they require orthogonality by an inner product defined in-sample or with respect to a given distribution

3 Likes

@danielwe it makes total sense. Thanks, it really helps a lot!

This functionality replica will form part of my undergrad thesis. Do you know any reference from which I can find the relationship between the QR decomposition and the polynomials which?

Thanks a lot btw!

I’m not familiar with the relevant literature, unfortunately. My previous post was just pieced together in my attempt to understand the poly docstring. The StackOverflow entry I linked, as well as the Wikipedia page on the Vandermonde matrix, were helpful in the process, but are obviously not the kind of primary sources you want for your thesis.

Anyway, the idea boils down to orthogonalizing the columns of the polynomial regression design matrix (the Vandermonde matrix) because orthogonal (uncorrelated) features give more robust and interpretable coefficients. Maybe one of the references listed on the Wikipedia page on polynomial regression contains what you need?

Did you look at the references in R’s poly docstring?

Chambers, J. M. and Hastie, T. J. (1992) Statistical Models in S. Wadsworth & Brooks/Cole.

Kennedy, W. J. Jr and Gentle, J. E. (1980) Statistical Computing Marcel Dekker.