# Sparse matrix error with forward diff

**URL:** https://discourse.julialang.org/t/sparse-matrix-error-with-forward-diff/108170
**Category:** General Usage
**Tags:** forwarddiff, sparse
**Created:** [December 30, 2023, 1:39am UTC](https://discourse.julialang.org/t/sparse-matrix-error-with-forward-diff/108170 "2023-12-30T01:39:08Z")
**Posts on this page:** 1
**Showing post:** 8

<div class="post-metadata">

### Author: ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [December 30, 2023, 9:54pm UTC](https://discourse.julialang.org/t/sparse-matrix-error-with-forward-diff/108170/8 "2023-12-30T21:54:27Z")

</div>

> [@rocco\_sprmnt21](#):
>
> _gradient(det(M(x))) = det(M(x))_ \* _trace(inv(M(x))_ \* _gradient(M(x)))_

The “gradient of a matrix” is not a thing, as I understand the gradient; this is not how the chain rule works with matrices. (ForwardDiff agrees with me: it gives a `DimensionMismatch: gradient(f, x) expects that f(x) is a real number.` if you try to compute `ForwardDiff.gradient(x -> M(x), x)` … your code just didn’t get that far.)

A more explicit version that is correct is:

\frac{\partial \log \det M(x)}{\partial x\_k} = \mathrm{trace} \left[M^{-1} \frac{\partial M}{\partial x\_k} \right]

Yes, this still involves M^{-1} applied to the derivative of M, which will in general be dense. But there are various things that you can do depending on the structure of M. (e.g. if \partial M / \partial x\_k is sufficiently sparse, or perhaps low rank, you may only have to apply `M \ ...` to a few vectors. Or you can use iterative trace-estimation techniques. Or …) When matrices get big and sparse, you often need to think about the structure of your specific problem carefully (which is one of the difficulties with making AD fully automatic for large sparse-matrix calculations).

(I just ran into a [2019 preprint](https://arxiv.org/abs/1911.00685) specifically on efficient computation of the gradient of the log–determinant for large, sparse matrices that may be worth looking over.)

> [@Dan](#):
>
> using the Laplace Expansion (on the first column), gives something workable

But worse than exponential (factorial) complexity? If you are using sparse matrices, I’m assuming your matrices are huge. If you have small-ish matrices, you should use dense matrices anyway, and ChainRules.jl knows how to differentiate through log-determinants of dense matrices.

---

_[View the full topic](https://discourse.julialang.org/t/sparse-matrix-error-with-forward-diff/108170)._