3

Which is the most used classifier applied once PCA is performed?

I computed PCA on my data, and now I have the points projected on a line, I was wondering what is the preferred classifier. Is a discriminant function for the normal density a good choice? Or is it better to use SVM?

amoeba
  • 93,463
  • 28
  • 275
  • 317
giuseppe
  • 545
  • 3
  • 12
  • See [this](http://stats.stackexchange.com/questions/54547/should-pca-be-performed-before-i-do-classification), [this](http://stats.stackexchange.com/questions/50932/does-pca-followed-by-lda-make-sense) and [this](http://stats.stackexchange.com/questions/52773/what-can-cause-pca-to-worsen-results-of-a-classifier) as well. – learner Jan 01 '14 at 14:07

2 Answers2

6

Cyborg's answer is basically right, but perhaps a little too strong.

Classification and PCA are not linked particularly strongly. People sometimes use PCA to reduce the dimensionality of their data set before classification. This is optional and it's often done for practical reasons: they know a priori that the first PCs contain a signal and the rest are noise or their computing platform simply can't handle the full-rank matrix (less of an issue now, but maybe you're using a huge data set or are on embedded hardware).

Having done PCA, you're still free to do whatever you want next. Your choice of classification or clustering algorithm is totally unconstrained. Throw it into an SVM, pipe it through a naive bayes classifier, whatever--it's your choice. That said, some algorithms make a little bit less sense. For example, one of the major strengths of decision trees is that they produce an interpretable model and this might be trickier on the PC-transformed data

Matt Krause
  • 19,089
  • 3
  • 60
  • 101
2

If you use a classifier, then you generally don't use PCA before.

Classifiers don't require that you reduce the dimensionality of your data. Reducing dimensionality (via PCA) may discard important information.

If you didn't reduce the dimensionality, then PCA also makes little sense before classification, because you loose the original meaning of the features.

cyborg
  • 551
  • 3
  • 6
  • 1
    This suppose that PCA obliges you to work with fewer dimensions. That's a choice, not an obligation. – Nick Cox Jan 01 '14 at 11:35
  • Forgive my ignorance, but what should I do once I perform PCA? Apply the discriminant function for the normal density? Or there are better choices? Usually, what do people do after they apply PCA? – giuseppe Jan 01 '14 at 13:00
  • 1
    There is nothing special people do after PCA. I think that I don't understand your goal. – cyborg Jan 01 '14 at 14:17