# PosDefManifoldML: machine learning for positive definite matrices

**URL:** <https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620>\
**Category:** Package Announcements\
**Tags:** machine-learning\
**Created:** [December 23, 2019, 12:26pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620 "2019-12-23T12:26:10Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 23, 2019, 12:26pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/1 "2019-12-23T12:26:10Z")

</div>

Hello all,

_PosDefManifoldML_ is a Python-free package for classifying data in the [**Riemannian manifolds**](https://en.wikipedia.org/wiki/Riemannian_manifold) **P** of real or complex [**positive definite matrices**](https://en.wikipedia.org/wiki/Definiteness_of_a_matrix), for example, covariance matrices, Fourier cross-spectral matrices, kernels, etc. It is based on the [PosDefManifold.jl](https://github.com/Marco-Congedo/PosDefManifold.jl), [GLMNet.jl](https://github.com/JuliaStats/GLMNet.jl) and [LIBSVM.jl](https://github.com/mpastell/LIBSVM.jl) packages.

For the moment being, _PosDefManifoldML_ implements the Riemannian **Minimum Distance to Mean (MDM)** classifier, which operates directly in **P** , the **elastic net logistic regression** (including the pure **Ridge** and pure **Lasso** logistic regresison model) and several **support-vector machine** classifiers in the **tangent space**. The models operating in the tangent space can be used also for traditional (Euclidean) feature vectors and in a Riemannian context as here implemented have won several international machine learning competitions in the field of _brain-computer interface_ (see table 1, page 29 [here](https://hal.archives-ouvertes.fr/hal-01570120/document)).

> **[GitHub - Marco-Congedo/PosDefManifoldML.jl: A Julia Package for Machine...](https://github.com/Marco-Congedo/PosDefManifoldML.jl)**
>
> A Julia Package for Machine Learning on the Manifold of Positive Definite Matrices - GitHub - Marco-Congedo/PosDefManifoldML.jl: A Julia Package for Machine Learning on the Manifold of Positive Def...

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [December 23, 2019, 1:24pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/2 "2019-12-23T13:24:17Z")

</div>

Cool! Any plan to support the MLJ.jl interface?

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 23, 2019, 1:41pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/3 "2019-12-23T13:41:16Z")

</div>

That would be nice, if there is someone interested in helping with that.

---

<div class="post-metadata">

**Author:** ![mateuszbaran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mateuszbaran/32/221842_2.png) [@mateuszbaran](https://discourse.julialang.org/u/mateuszbaran)\
**Post date:** [December 23, 2019, 4:03pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/4 "2019-12-23T16:03:49Z")

</div>

Nice package. You might be interested in the [Manifolds.jl](https://github.com/JuliaNLSolvers/Manifolds.jl) package, with it you could do ML on many other manifolds (we have the SPD manifold with two metrics).

---

<div class="post-metadata">

**Author:** ![lwhitefox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lwhitefox/32/4363_2.png) [@lwhitefox](https://discourse.julialang.org/u/lwhitefox)\
**Post date:** [December 23, 2019, 5:01pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/5 "2019-12-23T17:01:05Z")

</div>

This looks interesting, and I appreciate the nicely written Readme - it would also be cool to have some references to publications on the topic as well to help fill in gaps for those of us looking for new tools!

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 23, 2019, 6:10pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/6 "2019-12-23T18:10:43Z")

</div>

Thanks. I am concentrating on the manifold of positive definite matrices, for which i implemented 10 metrics in [PosDefManifold.jl](https://github.com/Marco-Congedo/PosDefManifold.jl). The level of abstraction of [Manifolds.jl](https://github.com/JuliaNLSolvers/Manifolds.jl) seems really impressive though.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 23, 2019, 6:14pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/7 "2019-12-23T18:14:24Z")

</div>

Thank you. There are several references with the links to the pdf’s in the docs [here](https://marco-congedo.github.io/PosDefManifoldML.jl/dev/). Also, i wrote an extensive introduction [here](https://marco-congedo.github.io/PosDefManifold.jl/dev/introToRiemannianGeometry/).

---

<div class="post-metadata">

**Author:** ![lwhitefox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lwhitefox/32/4363_2.png) [@lwhitefox](https://discourse.julialang.org/u/lwhitefox)\
**Post date:** [December 23, 2019, 8:21pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/8 "2019-12-23T20:21:09Z")

</div>

Very nice - thanks!

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [December 23, 2019, 10:09pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/9 "2019-12-23T22:09:38Z")

</div>

Marco, your documentation is gorgeous! Fantastic work here.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 24, 2019, 8:51am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/10 "2019-12-24T08:51:00Z")

</div>

Thank you very much Casey, fantastic as well is your willing to read it. All the packages i release are documented alike, as i believe a good documentation is fundamental for making a package really useful. All details are covered there and, hopefully, reading the documentation is a pleasant and instructive experience.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [December 24, 2019, 1:18pm UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/11 "2019-12-24T13:18:57Z")

</div>

This looks like a quite interesting package, thanks for providing it with such a detailed documentation. Might I suggest a small criticism, I would like to comment that the manifold for illustrations looks like having positive curvature, while the SPDs with most metrics have nonpositive curvatures. That apart, the illustrations look really nice; with which programm are you doing these?

We could try to get more metric into [Manifolds.jl](https://github.com/JuliaNLSolvers/Manifolds.jl), where we currently have 3 metrics covered (more or less) – I might take a look and “steal” a few metrics (of course documenting the origin of course). Or with [ManifoldsBase.jl](https://github.com/JuliaNLSolvers/ManifoldsBase.jl) try to get your algorithms into other manifolds as well. Of course only if you’d be interested (we can of course help!).

PS: Thanks for referring to [Manopt.jl](https://manoptjl.org) in your references. To some extend your ML methods are also optimization methods in that regime.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [December 25, 2019, 4:29am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/12 "2019-12-25T04:29:39Z")

</div>

The package [DirectSum.jl](https://github.com/chakravala/DirectSum.jl) is intended to provide a potentially universal interface for metrics. Its current design is not adapted for use with matrices, since it was designed for [Grassmann.jl](https://github.com/chakravala/Grassmann.jl). However, it could be adapted for use with matrices. For diagonal metrics, it uses a bit encoding. However, the `VectorBundle <: Manifold` is not limited to bit encoded diagonal metrics, it can be extended for any quadratic form to define a metric, although I have not taken the time to fully do that yet. In my experience, something like `DirectSum` is very useful for working with pre-allocated metric algebras.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 25, 2019, 9:30am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/13 "2019-12-25T09:30:46Z")

</div>

Hello Ronny,  
indeed the curves on the figure seem like curves with positive curvature, however this makes much nicer pictures. As this is just for illustration purposes, I have been using this kind of figures since a long time. I am doing the art work simply with power point and [paint.net](http://paint.net) (in Windows).

I have implemented many metrics for completeness, but i don’t think all of them are necessary. For the manifold of positive definite matrices I would say that the Jeffrey and S-divergence, along with the Wasserstein metric (which has been recently shown to be a Riemannian metric), nicely complement the affine invariant metric and the Euclidean and log-Euclidean distances.

In [riemannianGeoemtry.jl](https://marco-congedo.github.io/PosDefManifold.jl/dev/riemannianGeometry/#Means-1) i have implemented gradient-descent iterative algorithms to estimate the center of mass with all metrics that require an iterative algorithm, including the whole family of power means and p-means (e.g., the median). Sure you can copy or call them in [Manopt.jl](https://www.manoptjl.org/stable/).

I would be glad to contribute. Indeed [PosDefManifoldML.jl](https://github.com/Marco-Congedo/PosDefManifoldML.jl) can be easily adapted to classify data on the tanget space of any manifold for which a projection on the tangent space is defined. As long as a function for computing distances and means on the manifold are ready to be called, classification on the manifold with _PosDefManifoldML.jl_ is also straightforward.

Good to see that there is a community working in Julia on Riemannian stuff. Some of my former students may be interested in contributing. I am confident people will join, slowly as they get convinced on how nice Julia is!

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [December 25, 2019, 9:45am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/14 "2019-12-25T09:45:38Z")

</div>

Hi Marco,  
oh, of course median and mean are the first examples to illustrate the solver framework, so they are already within [Manopt.jl](https://manopt.org), we even ported easier (without the framework) versions to [Manifolds.jl](https://github.com/JuliaNLSolvers/Manifolds.jl), together with variance estimation within the statistics we provide on arbitrary manifolds. But I definetly will take a look at the Wasserstein metric (affine, LogEucldiean and LogCholesky are already available). Which one do you mean with Euclidean? The metric from the embedding would make it complicated, since points not on the manifold have finite distance?

Thanks for the info on your illustrations, just curius, what other people use (I use TikZ for 2D and Asympotote for 3D); for the illustration – yes, the positive curvature makes the images nicer, maybe it’s a trade-off between exacteness and readability of the figure.

It’s great to see more people working on manifolds in Julia, yes! And exactely, as you write, your methods should be doable on any tangent space, so generalizing that would be great.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 25, 2019, 10:00am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/15 "2019-12-25T10:00:56Z")

</div>

Hi Ronny,

the algorithm i implemented for the Wasserstein barycenter is the fastest i have found in the literature so far.

By ‘Euclidean’ i mean just the Euclidean distance `||P-Q||`.

OK, i will check what it takes to allow passing a `ManifoldsBase.Manifold` to machine leaarning methods of _PodSefManifoldML.jl_. I will start a thread in the _Manifold.jl_ github.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [December 25, 2019, 10:11am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/16 "2019-12-25T10:11:49Z")

</div>

Then I will definetly take a look at that algorithm!

But then `Euclidean`is not a Riemannian metric on the symmetric positive definite matrices, since nonpositive-definite matrices have a finite distance.

Great, looking forward to reading about that thread to discuss that idea further.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [December 25, 2019, 10:19am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/17 "2019-12-25T10:19:45Z")

</div>

Sure, i allow using the Euclidean distance for compatibility with other staff. PosDefManifold actually is for manipulating positive definite matrices in general, not just on the Riemannian manifold.

---

<div class="post-metadata">

**Author:** ![kellertuer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kellertuer/32/220707_2.png) [@kellertuer](https://discourse.julialang.org/u/kellertuer)\
**Post date:** [December 25, 2019, 11:35am UTC](https://discourse.julialang.org/t/posdefmanifoldml-machine-learning-for-positive-definite-matrices/32620/18 "2019-12-25T11:35:14Z")

</div>

Thanks for the clarification. Then `Euclidean` is slightly misusing the idea of a metric, but maybe that’s the easiest way to model(include it in your package.