2

I have a bit of problems understanding how PCA and SVD works, as most materials focus on calculating the factors rather than the classification of new entries. In order to provide some context of my questions, let us first consider the following model:

I have some data set $X$ with $p$ features and $n$ rows (i.e. observations). Now suppose I use the first two factors $f_{1}$ and $f_{2}$ to create the following model $y=\beta_{0}+\beta_{1}\cdot f_{1} + \beta_{2}\cdot f_{2}$. Now suppose I get the following row vector $r^{*}$. How can I project $r^{*}$ onto $f_{1}$ and $f_{2}$?

A second question, as far as my understanding goes if we do SVD on matrix $X$ we get: $X=U\Sigma V^{*}$, but which are actually the eigenvectors? And are those eigenvectors the same as what I got with $f_{1}$?

Thank you for helping me understanding.

Snowflake
  • 1,075
  • 1
  • 10
  • 19

1 Answers1

3

The model you define is somewhat inaccurate unless $y$ is of dimensions $p \times 1$. Is it?

Having said that assuming the eigenvectors $[f_1 f_2]$ define a $p \times 2$ matrix you simply do the multiplication: $[f_1 f_2]^T r^*$ to get the new scores. This is equivalent of projecting $r^*$ to the axial system defined by $f_1$ and $f_2$.

Both $U$ and $V^*$ are unitary matrices. If the matrix $X$ was symmetric $U$ would equal $V^*$. If you want the eigenvectors that are the same as the ones returned by PCA you are looking to use $V^*$.

Please see the post by the user @amoeba found here regarding the relation between SVD and PCA. I believe it will aid your understanding a lot.

usεr11852
  • 33,608
  • 2
  • 75
  • 117