Hey, I’d like to get derivatives for a linear least squares problem where the parameter appears in the right hand side: using LinearAlgebra using ForwardDiff A = rand(10, 3) b = rand(10) qr_fact = qr(A) function f(p) res = b * p v = qr_fact \ res return v end g = ForwardDiff.deriva…

This is slightly simpler and marginally faster in my benchmarks: A1 = rand(10, 3) b1 = rand(10) function F(A, b, x) qr_fact = qr(A) m, n = size(A) res = b * x v = qr_fact.R \ (qr_fact.Q[:, 1:n]' * res) return v end g = ForwardDiff.derivative(x -> F(A1, b1, x), 0.1) benchmarki…

Tanks for the pointer. I’ve looked at this question, but I don’t need to differentiate qr() itself. I want to get derivatives for the least-squares solution x^*(p) = \operatorname{arg\,min}_x \lVert A x - b(p) \rVert_2, which I compute via QR decomposition as A is a constant matrix. That is, let…

@mohamed82008 now that ImplicitDifferentiation.jl supports ForwardDiff.jl, is this something that could be solved with DifferentiableFactorizations.jl ?

This works: using LinearAlgebra using ForwardDiff A = rand(10, 3) b = rand(10) struct QRMatrices Qt::Matrix{Float64} R::Matrix{Float64} function QRMatrices(A) qr_fact = qr(A) dim, n = size(A) qt = Matrix{Float64}(undef, n, dim) for j = 1:n …

DF.jl is for differentiating the output of a factorization wrt to its input. It seems the factorization in the OP is a constant rather than a function of the input so you shouldn’t need DF.jl.

Cannot ForwardDiff through linear least squares via QR

Specific Domains

jd-foster September 26, 2023, 12:18pm 2

There is an older but similar question here that might give some pointers: Using ForwardDiff on qr factorization

Topic		Replies	Views
Using ForwardDiff on qr factorization Optimization (Mathematical) linearalgebra , forwarddiff	11	923	July 7, 2020
`ForwardDiff` and `LAPACK` don't get along Optimization (Mathematical) optim , vscode , forwarddiff	2	587	April 2, 2021
Least squares in julia General Usage linearalgebra	7	850	September 26, 2023
Does svd() not support forwarddiff? General Usage question	6	748	July 23, 2022
QR decomposition with julia. How to? New to Julia question	10	3459	January 4, 2023

Cannot ForwardDiff through linear least squares via QR

Related topics