PosDefException, increasing loss of precision

General Usage
1

I’m trying to construct an mcmc sampler for a HMM model which has a Bayesian Linear Regression component. However, as the mcmc iteration progresses, i quickly run into PosDefException errors when trying to simulate a multivariate normal or inverse wishart. Example of error:
ERROR: PosDefException: matrix is not positive definite; Cholesky factorization failed.

Checks on my some of the matrices causing the error reveal that with each update, the values of the matrices get larger and larger, in what i think leads to increasing error in numerical precision. Hence, the minimum eigenvalues of the matrices also increases, up to the point that the tolerance is no longer acceptable for generating multivariate normal or inverse wishart random variables.

Any idea how I can circumvent this problem? for example by perturbing the matrices such that it is still representative enough and also within tolerance. Another possibility is that there is a bug in my algorithm leading to the big numbers and consequential loss of precision, but I wonder if there is a way to deal with the PosDefException issue as other bugs in the code are not apparent to me now.

2

If you know the matrix is (analytically) Hermitian you can pass Hermitian(A) to cholesky instead, which ignores the upper/lower part of the matrix and assumes it is Hermitian.

2 Likes
3

thanks @fredrikekre, i’ve actually done that but was still getting the errors. Although analytically i believe that the matrices are positive definite, as they should be for a Bayesian Multivariate Linear Regression. Any thoughts?

P.S. I’ve managed to find an unrelated bug that is causing the elements of the matrices to grow very fast and avoided the error.

4

It is unclear what is happening without code, especially the construction of this matrix. Ideally, it would be PD by construction, using a suitable transformation, then as @fredrikekre suggested you can just wrap it in a Hermitian.

However, if you can decompose the PSD matrix into a variance and a Cholesky factor of the correlations, you can also use TransformVariables.CorrCholeskyFactor.

If you are doing MCMC, this could be a warning sign for an ill-defined problem, eg an improper posterior.

1 Like
5

thanks for your tips again @Tamas_Papp. I should definitely look into my model specifications again.