0

PCA operates by the following principles/constraints

  • Principal components are orthogonal basis vectors spanning the original feature/variable set
  • Principal components maximize variance between observations

Questions:

1. Is there an analysis or manipulation of PCA to achieve the following?

  • Principal components are orthogonal basis vectors spanning the original feature/variable set
  • Principal components minimize variance between observations

2. Additionally what would be an intuitive interpretation of the principal component loadings/coefficients?

My interpretation is that for the highest principal components, they would represent original features that explain homogeneity in the observation set.

Chill2Macht
  • 5,639
  • 4
  • 25
  • 51
Brendan Frick
  • 101
  • 2
  • 1
    Why not just look at the last principal component[s]? – gung - Reinstate Monica Mar 24 '17 at 16:27
  • 1
    Here's a thread you may want to look at: http://stats.stackexchange.com/questions/2691/making-sense-of-principal-component-analysis-eigenvectors-eigenvalues (as for the second part of your question). – Stefan Mar 24 '17 at 16:28
  • @gung does the last principal component demonstrate minimal variance? Or does it represent satisfaction of the the orthogonality constraint, given the other components that are already maximized for variance? – Brendan Frick Mar 24 '17 at 16:34
  • It isn't guaranteed to be the minimum possible, but it will be very small. Do you really need that guarantee? – gung - Reinstate Monica Mar 24 '17 at 16:35
  • @gung I interpret the last PC to have almost nothing to do with variance between observations and everything to do with maintaining orthogonality between other PCs. In that sense the last PC is the least influenced by the data and is not a good representation for the observations. From `1,...,n` components, each component `k` represents the **maximal** variance combination of features that remains orthogonal with all principal components less than `k`. If that's true, then the later components are poor measures of estimating **maximal** variance, not good estimators of **minimal** variance – Brendan Frick Mar 24 '17 at 16:39
  • I signaled this is a duplicate not because the question is a duplicate, but because the answer I posted answers both your questions. It also explains the sense in which @gung is right. – whuber Mar 24 '17 at 16:44
  • It is based on maintaining orthogonality, & has no freedom to reorient, given the existing PCs. That is correct. It will nonetheless have less variance than any other. I wouldn't call it a good representation of the data, but the lowest variance vector through your dataset (if different from the last PC) would also be a poor representation of the data. I'm wondering if you really need to guarantee that a vector is the absolute minimum possible for your data, & why you might need that? Maybe the last PC is good enough. – gung - Reinstate Monica Mar 24 '17 at 16:44
  • @gung, thanks, I now realize that there are many basis states that would contribute to "almost no variation in the data" -@whuber. And the orthogonality of the the last PC is at least as representative of homogeneity than any other minimal-variance combination. In a heterogeneous data-set, the last PC (performed on a homogeneous subset) is more likely to contribute to describing the homogeneous subset because it **(a) does not contribute to variance** and **(b) is orthogonal to all observation bases that do contribute to variance** – Brendan Frick Mar 24 '17 at 17:00

0 Answers0