1

I have been learning about PCA and SVD. And I know that to standardize the features before PCA is necessary. But I came across the book rating matrix, which makes me confused about what standardization do to the features.

So in a book rating matrix, a row is the rating a user gives to different books, and it can be ranging say from 0 to 5. So in this matrix, we know that all columns have the same unit measurement (1). However, for each column, the variance is not necessarily 1 (think about a book with ratings [1,4,5,1,1,1]). But it seems that because all columns/features have the same measurement, we do not need to standardize this matrix before PCA.

This makes me wonder that when we perform standardization on a dataset with features on different measurement(age and income for example), we are bringing the variance to 1 for ALL the features, my question is that will this make the features lose information?

Or more generally, should unit measurement, or unit variance, be the goal of data preprocessing?

amoeba
  • 93,463
  • 28
  • 275
  • 317
ZEE
  • 31
  • 3

1 Answers1

1

In your book rating example, normalization would still be useful even if the units seem to be consistent. This is because each reader could have an inherent bias; for example, one reader could always give generally high ratings to books, while another reader could always give generally low ratings to books. Put another way, one reader's "3" rating may not be the same as another reader's "3" rating.

liangjy
  • 646
  • 4
  • 12
  • yeah, that makes sense. I think similar approach is also done in collaborative filtering recommender. So is that true that when it comes to scaling features, we would usually prefer to make all features have unit variance, aka 1? – ZEE Dec 18 '17 at 06:39
  • This answer makes sense but I would replace "would still be useful" by "could still be useful". In some cases it might make total sense to omit standardization. @ZEE. See here https://stats.stackexchange.com/questions/53 (e.g. an answer from lep). – amoeba Dec 18 '17 at 09:12