2

I understand that when we are doing PCA, we are choosing an axis where the data has the maximum amount of variance. Then, we are restricted to choosing only axes which are orthogonal to the first axis we chose.

That means that if we want to maximize some kind of average or sum of the variances of all axes that we chose, we may not get to an optimal solution with PCA, right?

From what I understand, PCA chooses the axis with most amount of variance at each step. So does that make it greedy?

amoeba
  • 93,463
  • 28
  • 275
  • 317
octavian
  • 909
  • 2
  • 11
  • 18
  • 2
    You are confusing PCA as a procedure with a particular algorithm to carry it out. The algorithm you describe is indeed greedy, as is evident, but that doesn't make it suboptimal or incorrect. – whuber Dec 22 '17 at 16:29
  • Why would it not be suboptimal? Is there a proof that it maximises the average/sum of variances for the axes chosen? – octavian Dec 22 '17 at 16:43
  • 1
    Yes--and such demonstrations have appeared in several posts here. Consider searching. The posts would be related to the use of PCA and/or SVD for approximating matrices. [Here's the most recent I could find](https://stats.stackexchange.com/questions/301532/) based on this search: https://stats.stackexchange.com/search?tab=newest&q=pca%20matrix%20optimal%20. – whuber Dec 22 '17 at 16:46

0 Answers0