What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math easier, or is it more like: if you don't do it, you'll get the wrong answer.
This is something I've been wondering about for a while.
One other question: when you demean the data, do you also need to normalize it (i.e., divide by standard deviation)?
You can certainly do it, and the calculations work.
But of course you get different results.
The difference is between $\mathbf X^T \mathbf X$, the variance-covariance matrix (which is proportional to $\mathbf X^T \mathbf X$ if $\mathbf X$ is mean centered) , and the correlation matrix (for mean centering + variance scaling).
Whether using the data without centering is sensible depends on your data and application:
E.g. for my spectroscopic data, the practical result of not centering is usually that the first component will be the mean (and thus cover multiplicative changes of the mean spectrum) as the changes within the data are usually small compared to the mean.
In this situation, doing a PCA without centering allows me to capture changes e.g. by intensity fluctuations in the light source/uneven illumination or due to slight defocusing. Often, I want to ignore those for my further analysis, and in that case it is good to have them "together" in the 1st component instead of spread out over several components. Note that the difference is not only that the center is "prepeded": changes in the data that can be described by multiples of the mean will be captured by the "center component" if the data is mean centered, these changes will be captured by other components.
However, if I want to quantitate concentrations, these changes are often needed and centering to the mean or another from a physico-chemical point of view even more suitable center (e.g. blank spectrum) would be preferred.
The other extreme would be the situation where the mean is negligible compared to the changes. In that case, the first components won't change much. And, of course, there is the intermediate range where you can neiter assume the center to dominate $\mathbf X^T \mathbf X$, nor assume it negligible: in that case the results may change very much.
