0

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix of SVD to be similar, yet they are not. I found that I need to flip and multip one column by -1. Pricnc[pe components could be calculated using either the eigendecomposition or SVD. Given that the eigendecomposition give different output from the SVD for the covariance matrix may yield different PCA output.

import numpy as np
import matplotlib.pyplot as plt 

A=np.array([[1,3,2],
       [4,8,2],
       [3,9,7],
       [22,11,17],
       [55,33,66]])


A=A-np.mean(A,0)
print(np.dot(A.T,A)/(A.shape[0]-1))
co=np.cov(A.T)
print(co)
#%%
[D,UI]=np.linalg.eigh(co)
[aa,bb,cc]=np.linalg.svd(co)
print (UI)
print(aa)

UI
array([[-0.15348562, -0.77609875, -0.61164768],
   [-0.85628748,  0.41338399, -0.30965372],
   [ 0.49316722,  0.47621886, -0.72801215]])

aa
array([[-0.61164768,  0.77609875, -0.15348562],
   [-0.30965372, -0.41338399, -0.85628748],
   [-0.72801215, -0.47621886,  0.49316722]])

aa[:,::-1]
array([[-0.15348562,  0.77609875, -0.61164768],
   [-0.85628748, -0.41338399, -0.30965372],
   [ 0.49316722, -0.47621886, -0.72801215]])
Kernel
  • 145
  • 6

1 Answers1

2

From the definition of an eigenvector $v$ and eigenvalue $\lambda$ $$ \begin{align}Av &=\lambda v \\ A(-v) &= \lambda (-v) \end{align}$$ we can see that eigenvectors are only identified up to a nonzero multiple. Your discovery that multiplying by $-1$ makes the results coincide is not surprising, and amounts to a difference in how the two methods work.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • thank, any idea how to explain mismatch in only part of the matrix (one column) and not the whole matrix. I added the values of the matrix that I get above. Thanks – Kernel Mar 20 '19 at 23:52
  • 1
    @kernel One column is an eigenvector. My answer says that the sign and magnitude of an eigenvector is *arbitrary.* – Sycorax Mar 20 '19 at 23:56