Eigen values and vectors sorting

General Usage
, , ,
1

I am trying to work out how to sort eigen values and eigen vectors that are consistent with other languages. I use the following code on a covariance matrix:

eigen_vals, eigen_vector = eigen(covariance_matrix)

This gives

eigen_vals
4-element Vector{Float64}:
 0.00016396250742156755
 0.001731845741519984
 0.16021456116230823
 3.8378896305887458
eigen_vector
4×4 Matrix{Float64}:
  0.190256  -0.647716  -0.544303   -0.497997
 -0.711804   0.405559  -0.267156   -0.507429
  0.660267   0.549362   0.0438673  -0.51022
 -0.145554  -0.337915   0.794001   -0.483931

In Matlab, I get the following results:

eigen_vals 
  3.8379         0         0         0
         0    0.1602         0         0
         0         0    0.0017         0
         0         0         0    0.0002
eigen_vector
    0.4980    0.5443   -0.6477    0.1903
    0.5074    0.2672    0.4056   -0.7118
    0.5102   -0.0439    0.5494    0.6603
    0.4839   -0.7940   -0.3379   -0.1456

So basically, the eigen values are sorted in descending order and eigen vectors are also negated in some cases. How can I sort and negate eigen values and eigen vectors so that I get answer that is consistent with MATLAB?

2

To get the eigenvalues in the same order as in matlab, take
eigen_vals[end:-1:1], and eigen_vector[:, end:-1:1]. You cannot predict the sign of a column, because if v is an eigenvector corresponding to the eigenvalue \lambda, then Av=\lambda v. But to an eigenvalue does not correspond a single vector, but a subspace. Together with v, \alpha v (\alpha a scalar) is also an eigenvector corresponding to \lambda. Hence for both \alpha=1 and \alpha=-1 you get an eigenvector for the same eigenvalue.

1 Like
3

Thank you. I think in MATLAB, if all elements of eigen vector are negative then MATLAB changes the sign to positive. So just the first eigen vector has all negatives, so MATLAB returns all positives

4

For the eigenvectors you have roughly three choices:

  1. Try to figure out if you can find a system in how Matlab chooses its signs and adjust Julia’s eigenvectors to match.
  2. Design your own convention for choosing the signs and adjust both Matlab’s and Julia’s eigenvectors to match.
  3. Accept that the eigenvector signs are arbitrary and make sure that the following algorithm steps are invariant to whatever choice happened to be made.

The last option is by far the most sound. Notice also that when you have multiple equal eigenvalues the arbitrariness of choice from the subspace spanned by the corresponding eigenvectors is much larger than just a choice of sign.

2 Likes
5

But I really don’t undestand why you want the same eigenvectors like in Matlab. If \lambda is multiple of order 2, and the corresponding eigenspace is two dimensional, then you get two eigenvectors corresponding to \lambda, which form a basis in this subspace. But there exists an infinity of such bases. If the matrix is symmetric the basis is orthonormal. Matlab could return two orthonormal eigenvectors (v, w), while Julia another pair (t,s). Hence you cannot find necessarily a correspondence between the eigenvectors recturned by the two languages.

1 Like
6

Sorting out the eigen values in descending order and the corresponding eigen vectors was important to me. I was not sure if Julia is using a different sign convention and order of eigne values than MATLAB and Python. Your answer has helped me with my question

7

Or better: reverse!(eigen_vals) and reverse!(eigen_vector, dims=2)
(Assumming that you don’t mind to modify the output originally given by eigen. Otherwise you can use reverse instead of reverse!.)

2 Likes
8

Note that the eigen function takes a parameter sortby that is a function specifying how you want them to be sorted. If you want them sorted in descending order by the real part, you can just pass sortby=-:

julia> A = [i^2 + i*j + j^2 for i=1:4, j=1:4]
4×4 Matrix{Int64}:
  3   7  13  21
  7  12  19  28
 13  19  27  37
 21  28  37  48

julia> eigen(A, sortby=-)
Eigen{Float64, Float64, Matrix{Float64}, Vector{Float64}}
values:
4-element Vector{Float64}:
 97.34671140716996
  0.11020703271515231
  3.552713678800501e-15
 -7.456918439885142
vectors:
4×4 Matrix{Float64}:
 -0.260069  -0.629181   0.223607  -0.697492
 -0.374035   0.436083  -0.67082   -0.468966
 -0.526594   0.519101   0.67082   -0.0568572
 -0.717747  -0.380128  -0.223607   0.538835

Or if you wanted them in descending order by magnitude, you could pass sortby = λ -> -abs(λ).

This is usually more convenient than re-ordering them afterwards.

3 Likes