10

I have a training set of data on which I do principal components analysis (PCA) and save the loadings/eigenvectors/coefficient matrix. I want to use the eigenvectors to transform my testing data into the same principal component space, I know I just do matrix multiplication between the test data and the eigenvector matrix and there are other posts that explain this.

However, I calculate the PCs from the training data after centering the data so the mean is zero (I call this zero-centering). My question is this: how do I handle zero-centering the testing data before the matrix multiplication? Do I just subtract the means of the training data as I did to zero-center the training data? It would seem this is correct, since the other option I imagine is to use the mean of the testing data (in the case the testing data consist of a single instance, it is a 0 vector then), but maybe there are other options I am overlooking?

Can someone back me up that I just subtract the means of the training data from the test data and then multiply by the eigenvector matrix? Or refute me? Ideally provide a reference?

amoeba
  • 93,463
  • 28
  • 275
  • 317
PatEugene
  • 203
  • 2
  • 5

3 Answers3

12

Do I just subtract the means of the training data as I did to zero-center the training data?

Yes.

You are supposed to do to the test data exactly the same transformation that you did to the training data; this includes centering -- it should be done using the mean values obtained on the training set. If you standardized the training set, then you would also divide your test set by the standard deviations obtained on the training set. After that, you can project your test set onto the PCs of the training set.

amoeba
  • 93,463
  • 28
  • 275
  • 317
  • Thanks for the answer sir! ... I have a question please, how should the mean images computed over the dataset? shall I compute a single mean over (testing+training data) or shall I compute two different means one for training and the other for testing ? – Ibrahim Amer Sep 11 '16 at 13:36
  • Just a note, the mean you should remove from the test set should be the mean of the training data I believe right? To make the same transformation. Otherwise if sets are too different, means would be also different, and therefore the transformation applied to the test set, also different. Could anyone back me up? @amoeba? – ouranos Jul 02 '17 at 21:06
  • 1
    @ouranos That's right -- you subtract the mean of the training data from the test data. That's what I meant by "both transformations should be done using the values obtained with the training set." – amoeba Jul 02 '17 at 21:16
2

You need to subtract the mean of the training set from the test set and then take the projection on the eigenvectors. You should not take the mean of (training + test) set.

Also refer to Andrej Karpathy's notes here: http://cs231n.github.io/neural-networks-2/

Shubham Bansal
  • 156
  • 2
0

PCA computes the eigenvectors of the Covaraince matrix. The Covariance matrix uses an implicit centering of data. So it does not really matter whether you center your training data or not. The resulting eigenvectors and eigenvalues will be the same. This means you don't really have to center your test data. The projections that you obtain would only be in a different(tranlated by the mean) co-ordinate system if you center the test data.

Leena Suresh
  • 1