1

One way of viewing PCA is a reflection and rotation of the original coordinates, while keeping the points pairwise fixed.

For exposition purposes, take a data set in two dimensions with x and y positively correlated, this could result in a PCA with a 45 degrees anti-clockwise rotation.

It is the reflection part I do not understand. What do the reflections represent? What information does a reflection have that differs from the information content of a rotation? Wha is the difference between a reflection and the eigenvector directions?

Single Malt
  • 504
  • 1
  • 5
  • 15
  • 1
    Euclidean geometry teaches us that all orthogonal transformations are products of reflections--this isn't a statistical result. See https://en.wikipedia.org/wiki/Reflection_(mathematics). I cannot understand what you mean by "vectors which point to plus infinity," because there aren't any such things. Could you clarify this? What do you mean by a plot of "this principal component of $x$ against $x-1$"? Its meaning and construction are unclear--perhaps you could post an example? – whuber Aug 12 '20 at 16:47
  • This is two totally different questions. Please choose one of them to ask (or ask them in two separate posts). – whuber Aug 12 '20 at 19:20
  • Thank you for clarifying your questions. This one ("what do the eigenvector directions represent") is answered at https://stats.stackexchange.com/questions/2691/making-sense-of-principal-component-analysis-eigenvectors-eigenvalues. "Reflections" do not refer to the directions of the eigenvectors: they refer to the geometric process of rotating the principal axes of the ellipsoid of second moments. They may be a red herring, because you don't even need to conceive of the transformation as involving reflections at all: see https://stats.stackexchange.com/a/71303/919 for an investigation in 2D. – whuber Aug 12 '20 at 19:43
  • @whuber Both links have excellent answers. By second moments do you mean collectively the correlation and covariance matrices? – Single Malt Aug 12 '20 at 20:00
  • 1
    The covariance matrix. (The correlation matrix is derived from that and provides no additional information.) The covariance matrix is the second moment tensor, in physical parlance. In mechanics the first approximation you make is to represent a solid body by a point at its center of mass. That's *first order.* When you need to improve on that, the next approximation treats the body as an ellipsoid determined by the second moment tensor: *second order.* There are higher-order approximations, too. PCA, in some analogous ways, carries out the same second-order approximation to a distribution. – whuber Aug 12 '20 at 20:05

0 Answers0