3

https://stats.stackexchange.com/a/3374/92071 - In PCA, the components are actual orthogonal linear combinations that maximize the total variance. In FA, the factors are linear combinations that maximize the shared portion of the variance--underlying "latent constructs".

Now, I understand that eigenvalues represent the amount of variance captured by a particular dimension. In order to obtain these directions, wouldn't one be maximising the co-variance terms along with the variance ones, implicitly?

Varimax rotation (for FA) maximises only the co-variance terms irrespective of the total variance associated with the newly formed dimension. Is this an accurate difference between the two kinds of rotation?

Community
  • 1
Hardik Shah
  • 31
  • 5
  • 2
    Varimax or similar rotation of loadings has nothing to do with the difference between PCA and FA: http://stats.stackexchange.com/q/612/3277 – ttnphns Dec 19 '16 at 08:04
  • 2
    [One](http://stats.stackexchange.com/q/94048/3277), [two](http://stats.stackexchange.com/q/95038/3277). PCA accounts for covariance too, because variables co-variate, i.e. share variances. PCA however does not chase after the specific aim to explain the correlatedness as precise as possible. FA does. `In FA, the factors are linear combinations` Nope, factors are not linear combinations of variables. Factor _scores_ are. – ttnphns Dec 19 '16 at 08:07
  • 1
    PCA tends to approach, in results, to FA, as the number of variables grows. See [thread](http://stats.stackexchange.com/q/123063/3277). The two analyses are nevertheless theoretically distinct. – ttnphns Dec 19 '16 at 08:11
  • What I infer from your comments is, PCA will look for a direction that maximises variance in projections of multiple variables (this implicitly accouts for both variance and covariance). While FA cares only about the covariance space irrespective of individual variances. Correct? – Hardik Shah Dec 19 '16 at 09:00
  • 2
    Yes. FA "takes interest" in off-diagonal elements of the matrix. PCA does not, it however happens to account for them to an extent because covariances are variances shared: explaining total variance you cannot skip explaining shared variance. I recommend you to look through the links I gave. – ttnphns Dec 19 '16 at 09:16
  • 1
    `Varimax rotation (for FA) maximises only the co-variance terms irrespective of the total variance...` In [this answer](http://stats.stackexchange.com/a/193023/3277) I gave an illustrative chart where you can see the essence of PCA-as-rotation along as Varimax rotation. Both are just orthogonal rotations so formally they are comparible. But they optimize different goal. PCA hunts after maximal varince of _data points_ . [Varimax](http://stats.stackexchange.com/a/185245/3277) seeks to maximize variance of squared _loadings_ of variables for each factor. It doesn't have any aim wrt data cases. – ttnphns Dec 28 '16 at 07:48

0 Answers0