0

What is the correct sequence to perform PCA + cross-validation out of these two?

  • Perform PCA and then perform k-fold cross validation on the reduced features.
  • Perform k-fold cross validation in outer loop and perform PCA internally.
amoeba
  • 93,463
  • 28
  • 275
  • 317
Hellboy
  • 181
  • 3
  • 8
  • 1
    Clarification: do you mean that PCA is used to reduce dimensionality of the predictors that are later used to predict something else (e.g. in a regression or classification setting)? – amoeba Dec 01 '16 at 17:28
  • Just for the record: the second option is correct (as described in the linked thread). – amoeba Dec 01 '16 at 19:31
  • @amoeba, not necessarily. Since PCA is done on the features, not the predictions, even if your data have unknown outcomes, you can include them in your PCA to get a better approximation of your PCs, since you can do this same thing when you built your model on data with unknown responses as well. You only would have to do PCA in each CV fold if the model you are creating has to be made in advance of knowing what data it would be tested on. – Barker Dec 05 '16 at 19:08

0 Answers0