# Differentiating random walk probability w.t.r. rate of jump

**URL:** https://discourse.julialang.org/t/differentiating-random-walk-probability-w-t-r-rate-of-jump/107116
**Category:** Probabilistic Programming
**Tags:** autodiff
**Created:** [December 4, 2023, 2:46pm UTC](https://discourse.julialang.org/t/differentiating-random-walk-probability-w-t-r-rate-of-jump/107116 "2023-12-04T14:46:10Z")
**Posts on this page:** 1
**Showing post:** 14

<div class="post-metadata">

### Author: ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [December 20, 2023, 1:54pm UTC](https://discourse.julialang.org/t/differentiating-random-walk-probability-w-t-r-rate-of-jump/107116/14 "2023-12-20T13:54:44Z")

</div>

> [@aner-sanchez](#):
>
> An I don’t think automatic differentiation is the way to go with matrix exponentials.

You can do deterministic differentiation of matrix exponentation with a variety of algorithms, e.g. ChainRules.jl implements the algorithm in [Al Mohy et al. (2009)](https://eprints.maths.manchester.ac.uk/1218/). (And, in particular, if you eventually are computing a scalar-valued function, e.g. an ML “loss” function, with the matrix exponential as an intermediate step, then the chain rule can be carried efficiently through the matrix exponential.)

Part of the confusion in the math.stackexchange question you linked is how to think about what the derivative _is_, and in particular the proposed answer \frac{\partial \exp(DA)}{\partial (DA)} = DA \exp(DA) is a [category error](https://en.wikipedia.org/wiki/Category_mistake): it’s not even the right _kind_ of object for a matrix derivative. What you need is a Fréchet derivative, a linear operator on matrices, which is _not_ simply a matrix of the same size. See e.g. our [matrix calculus course](https://ocw.mit.edu/courses/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/), or [this short talk](https://www.youtube.com/watch?v=-l7JHalBubw) on the nature of the problem.

---

_[View the full topic](https://discourse.julialang.org/t/differentiating-random-walk-probability-w-t-r-rate-of-jump/107116)._