0

I am working with PCA to do dimensionality reduction to a set of data.

I have 1600 data points with 36 variables and I want to have a matrix with a new data set with 6 principal components. I managed to do it in matlab, so I have:

xtrain (normalized) <1600x36>

and I write:

coeff=pca(xtrain,'NumComponents',6)

which gives me a <36x6> matrix

So far so good. Now I want to know how to reverse the process. If I'm given a data set of how do I use the coeff matrix to pass it to a "n x 36" representation?

I would like to do this in matlab but a simple explanation can help me.

ttnphns
  • 51,648
  • 40
  • 253
  • 462
lesolorzanov
  • 109
  • 5
  • Do you want to get the original dataset if you are given the PCA vectors and the projected points? – Budhapest Sep 12 '13 at 01:15
  • Yes. But unfortunately I only have some of them not all the principal components – lesolorzanov Sep 12 '13 at 05:15
  • 1
    `Now I want to know how to reverse the process. If I'm given a data set of how do I use the coeff matrix to pass it to a "n x 36" representation?` this whole passage isn't clear. Give more detailed question, please. – ttnphns Sep 12 '13 at 05:56
  • 1
    Consider starting your research at http://stats.stackexchange.com/questions/2691/making-sense-of-principal-component-analysis-eigenvectors-eigenvalues so that you can appreciate what PCA does and what the limitations might be. (There's no way you can even approximate the original dataset from the information given. You could do a rough job if, in addition, you had the eigenvalues or their equivalents along with the six components you have retained.) – whuber Sep 12 '13 at 06:03

2 Answers2

0

PCA is used for dimensionality reduction. After projecting into fewer number of dimensions, you lose some variance (along the discarded eigen vectors). Hence it is impossible to recover the original signal.

Consider the following example. Let the original dimensions be $\hat{i},\hat{j},\hat{k}$ and a point be $(a_i\hat{i},a_j\hat{j},a_k\hat{k})$. Let us discard the last 2 eigen vectors and keep only the first one. Let the eigen vector be given by $(e_i\hat{i},e_j\hat{j},e_k\hat{k})$. Projecting the point on this eigen vector gives us the magnitude of the point along this direction which is $|P|=(a_i*e_i+a_j*e_j+a_j*e_j)$. Since the other two eigen vectors are discarded, we do not know its projections along those dimensions.

If you want to project this point back to the original space, you would get $(|P|\hat{i},|P|\hat{j},|P|\hat{k})$. Notice how just the above is different from the original point. If you had the other eigen vectors as well, they might add to the above.

Budhapest
  • 551
  • 1
  • 5
  • 12
  • Thank you very much. Now I understand why I couldn't reconstruct the original. But isnt true that sometimes the discarded eigenvectors can be so unimportant that the reconstructed data resembles a bit more to the original? What properties should the original data have to accomplish this? – lesolorzanov Sep 12 '13 at 16:29
  • In the 2 d case, suppose all the points lay on the line y=x. Then the first eigen vector will be the line y=x. There will not be any other eigen vector since there isn't any unexplained variance. Or if the values were close to 0, then the loss of variance will also be low – Budhapest Sep 12 '13 at 19:18
  • Your last formula seems to be wrong: why would a reconstruction have the same value in all three coordinates? – amoeba Feb 14 '15 at 22:44
-1

Rather than using PCA, you might want to look into using an autoencoder if reconstructing the data is necessary. Autoencoders are basically neural networks that can do dimensionality reduction, and if necessary, "decode" the original encoding. The decoded data will still be corrupted, but in all likelihood it will be better than trying to reconstruct data from PCA.

http://deeplearning.net/tutorial/dA.html

user62117
  • 112
  • 5
  • Thankyou! Any ideas how to do this a bit simpler in MATLAB? – lesolorzanov Sep 12 '13 at 16:27
  • @Zloy Sorry, I'm not familiar with what kind of neural net packages are available on matlab. Here is a tutorial from Andrew Ng's class on sparse autoencoders. http://www.stanford.edu/class/cs294a/sparseAutoencoder.pdf – user62117 Sep 12 '13 at 19:25