1

I have some data about the $\{noise,~ size,~ speed,~ length,~ width\}$ of cars. I have performed SVD, and I want to reconstruct my data using only the first 2 principal components.

I subtracted mean from data and then did

[U,S,V] = svd(matrix,0) in matlab

If I am right the first 2 columns of V would be the first 2 principle components can anyone help me reconstruct the data please.

also my data is originally a $14 \times 5$ matrix

Peter Leopold
  • 1,653
  • 7
  • 21
Coder
  • 13
  • 5

1 Answers1

0

Let $X$ be a $14\times5$ matrix of 14 data points with 5 factor values per point. After subtracting the column means from each matrix element in the respective column, we are left with the zero-column-centered matrix $X_0$.

The eigenvectors of the $5\times 5$ covariance matrix $X_0^TX_0$ are the principal components, and as you indicate, they are also the column vectors of $ 5\times 5 $ matrix $V$ arising from the singular value decomposition of $X$, viz., $X=U\Sigma V^T.$

Your data came to you in the original basis, which we make explicit as follows $X_0=X_0^{orig}.$ Now you seek to write your zero-centered $X_0^{orig}$ in the PCA basis $X_0^{PCA}$. The change of basis is accomplished by

$$ \begin{aligned} X_0^{PCA} &= (V^T X_0^{orig,T})^T = X_0^{orig} V \\ (14\times5)&= ((5 \times 5) (5 \times 14))^T= (14\times5)(5\times5).\\ \end{aligned} $$

That is how you project the data matrix onto all five principal components and report the projections in the PCA basis.

If you want to project the original matrix $X$ into the first two principal components and leave it in the original basis, you simply set the eigenvalues associated with the other components to zero.

$$ \Sigma' = \begin{cases} \Sigma_{ii}, &i=1,2 \\ 0, &i=3,4,5 \\ \end{cases} $$ So SVD will deliver the projected data

$$ X'_0 = U\Sigma'V^T. $$

This reconstructs $X'_0$. To return it to the original matrix, don't forget to add the means back to the columns!

This was all nicely discussed in @amoeba's link, which I just noticed. :)

Peter Leopold
  • 1,653
  • 7
  • 21
  • is V not (5x2) since we are only interested in the first 2 principle components meaning only 2 columns of V? – Coder Mar 13 '19 at 00:12
  • You asked for a (full) reconstruction of the data. In that case, $V$ has to be $5 \times 5$. But if you are only interested in a partial reconstruction consisting of projections on the first 2 components then, sure, you may cut off the last 3 columns and transform it into $V^*$, a $5 \times 2$ matrix. – Peter Leopold Mar 13 '19 at 00:16
  • i think i figured it out: i used U*S which gives a (14*5) so i changed it so that i use only (14*2) of U*S and then i got the V (pc) and took the first 2 columns which gives (5*2) and i transposed this to get (2*5) and basically did US(14x2) *(2x5)Vtranspose i get a matrix with values quite similar to the original matrix... would this be right? – Coder Mar 13 '19 at 00:21
  • 1
    Oh, I see what you are looking for. OK, I've changed the my answer accordingly. – Peter Leopold Mar 13 '19 at 02:14