R PCA: principal (psych package) vs prcomp loadings

Question

I have a couple of questions regarding differences in loading values when using prcomp and principal (from the psych) package to perform PCA

When conducting PCA using prcomp:

pca_results <- prcomp(df, center = TRUE, scale. = TRUE)
pca_results$rotation[,1:5]

I get the following loadings:

              PC1          PC2           PC3           PC4           PC5
q3   -0.016809164  0.134292686 -0.1757822345  1.108893e-01 -0.1319508350
q5    0.050866015 -0.161877460  0.0892043331  2.767157e-02  0.1474154691
q8   -0.008870246 -0.015767530  0.0115132365  1.618538e-01  0.2722705733

When using principal from the psych package:

pca_fit <- principal(df, nfactors = 5, rotate = "none")
pca_fit$loadings

I get these loading values instead:

     PC1    PC2    PC3    PC4    PC5   
q3          -0.335 -0.369  0.207  0.211
q5   -0.149  0.403  0.187        -0.235
q8                         0.301 -0.435

My first question: why is there a difference in the loading values between the 2 methods? Which is correct? I've also tried prcomp with center and scale set to FALSE but the numbers still don't match

My second question: what does the missingness in the principal loadings indicate? Is it some threshold of loading value, below which nothing is shown?

Please do not mix "loadings" with "eigenvectors". `prcomp` improperly calls eigenvectors "loadings". Search this site for `PCA loadings eigenvectors`, to learn the difference. — ttnphns, Feb 18 '17 at 06:23

dk14 · Accepted Answer · 2018-02-10T06:51:11.390

As I can see from your data - the difference is only in scaling. For instance, PC3 is scaled with $psych = 2.099188243053083 * prcomp$, some scaled with negative number. So, both algorithms are correct as principal direction doesn't change with positive scaling, and negative scaling encodes the same space.

In order to see the whole picture - you can check out eigenvalues (from sdev - a squared root of eigenvalue):

pca_results <- prcomp(df, center = TRUE, scale. = TRUE)
> pca_results$rotation[,1:4]
                    PC1         PC2        PC3        PC4
Sepal.Length  0.5210659 -0.37741762  0.7195664  0.2612863
Sepal.Width  -0.2693474 -0.92329566 -0.2443818 -0.1235096
Petal.Length  0.5804131 -0.02449161 -0.1421264 -0.8014492
Petal.Width   0.5648565 -0.06694199 -0.6342727  0.5235971

> pca_results$sdev
[1] 1.7083611 0.9560494 0.3830886 0.1439265

Compare it with psych's loadings ("eigenvalues" are called SS-loadings here):

> pca_fit <- principal(df, nfactors = 4, rotate = "none")
> pca_fit$loadings

Loadings:
             PC1    PC2    PC3    PC4   
Sepal.Length  0.890  0.361 -0.276       
Sepal.Width  -0.460  0.883              
Petal.Length  0.992                0.115
Petal.Width   0.965         0.243       

                 PC1   PC2   PC3   PC4
SS loadings    2.918 0.914 0.147 0.021
Proportion Var 0.730 0.229 0.037 0.005
Cumulative Var 0.730 0.958 0.995 1.000

You can see that they're same (after squaring). So the algorithms output same factors, they just represent principal directions differently: eigenvectors in prcomp (unit length), loadings in psych (non-unit length).

So, the only "problem" with psych is that its vector doesn't have unit length. The prcomp vector has length 1, for instance for PC1:

prcomp: 0.5210659 * 0.5210659 + 0.2693474 * 0.2693474 + 0.5804131 * 0.5804131 + 0.5648565 * 0.5648565 = 0.99999992627343

psych: 0.890 * 0.890 + 0.460 * 0.460 + 0.992 * 0.992 + 0.965 * 0.965 = 2.918989

P.S. Yes, psych package doesn't show values less than epsilon, that's why there's some empty cells there. Edit: As @William Revelle pointed out "the reason the small loadings are dropped is that the loadings object is of class "loadings" and R print drops values less than .3. Unclass the loadings with (unclass(pca_fit$loadings) and then print them."

As a matter of fact, loadings in psych are not unit eigenvectors because it uses them for factor rotation. That's why loadings are eigenvectors scaled by the square roots of the respective eigenvalues (even if you specify no rotation). See https://stats.stackexchange.com/a/137003/137805

ok thanks. Finally, is epsilon some sort of special value in PCA? Why is that chosen as the threshold? — Simon, Feb 18 '17 at 04:15
no, I think psych is using it just to improve readability; coz psych is less "scientific" I'd say :). Another instance is that eigenvectors should have unit length, psych's vectors don't have it - you have to normalize them (rescale), prcomp gives you vectors with unit length by default - that's why you see difference in results. — dk14, Feb 18 '17 at 04:20
It is better to avoid calling eigenvectors (unit-normalized principal directions) "loadings" altogether. There is no "load" in them. http://stats.stackexchange.com/a/143949/3277 — ttnphns, Feb 18 '17 at 06:42
@ttnphns Thanks, I corrected the answer to highlight "principal directions" term — dk14, Feb 18 '17 at 07:01
Actually, the reason the small loadings are dropped is that the loadings object is of class "loadings" and R print drops values less than .3 Unclass the loadings (unclass(pca_fit$loadings) and then print them. Better yet is to actually print the pca results which shows the loadings without deleting the small values. — William Revelle, Feb 09 '18 at 22:10
@WilliamRevelle Thanks, I highlighted your suggestion in the edit. — dk14, Feb 10 '18 at 06:51

R PCA: principal (psych package) vs prcomp loadings

1 Answers1