1

In relation to PCA which is usually used in a setting where one wishes to maximize the variance as well as a reduction in dimensionality thereby choosing the eigenvalues with corresponding eigenvectors which explain most of the variance (i.e 90 %). Now, I get that there are general limitations to this process such if the data is not linearly correlated...etc, but is there specific problems more so associated with instead choosing to instead minimize variance (choosing the smallest eigenvalues, with corresponding eigenvectors)? A real life example might be minimizing financial risk to take an example. Since large variances are associated with low covariance, so I'm thinking that choosing small variances makes interference a bigger problem perhaps? Might this also be related to overfitting? I hope my question makes sense. Have not studied much about PCA so far.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
PianoMath
  • 11
  • 1
  • My answer at https://stats.stackexchange.com/a/74328/919 details one application. Your thoughts about overfitting sound related to the question about small PCA components at https://stats.stackexchange.com/questions/444545. – whuber Dec 10 '20 at 14:24
  • 1
    Some similar Qs of interest: https://stats.stackexchange.com/questions/87198 https://stats.stackexchange.com/questions/314562/pca-how-to-select-eigen-vectors-corresponding-to-small-eigenvalues-for-regressi – kjetil b halvorsen Dec 10 '20 at 14:29
  • 1
    Thanks, the provided links are very helpful indeed. I still want to investigate further which conditions makes it not possible/error prone to choose the smallest eigenvalues when performing PCA. – PianoMath Dec 11 '20 at 04:26

0 Answers0